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Preface 

The  synergy  between  logic  and  computational  complexity  has  gained  impor¬ 
tance  and  vigor  in  recent  years,  cutting  across  areas  such  as  proof  theory,  finite 
model  theory,  computation  theory,  applicative  programming,  database  theory, 
and  philosophical  logic.  This  volume  is  the  outcome  of  a  Workshop  on  Logic 
and  Computational  Complexity  (LCC),  organized  to  bring  together  researchers 
in  this  growing  interdisciplinary  field,  so  as  to  foster  and  enhance  collaborations 
and  to  facilitate  the  discovery  of  conceptual  bridges  and  unifying  principles. 

The  workshop  was  held  at  the  Indiana  University  Conference  Center  in  In¬ 
dianapolis,  from  the  13th  to  the  16th  of  October,  1994.  Forty-one  talks  were 
presented  at  the  meeting.  The  papers  in  this  volume  are  revised  versions  of  the 
workshop  papers,  or  closely  related  to  them.  Jean- Yves  Girard,  Stephen  Lindell 
and  Anil  Seth  were  unable  to  attend,  but  included  their  intended  contribution 
in  this  volume.  On  the  other  hand,  a  number  of  speakers  could  not  contribute  to 
this  volume,  for  various  reasons:  Maria  Bonet,  Philip  Bradford,  Kevin  Compton, 
Stephen  Cook,  Alexandar  Ignjatovic,  Neil  Immerman,  Phokion  Kolaitis,  Harry 
Mairson,  Yiannis  Moschovakis,  Jim  Otto,  Toni  Pitassi,  Andre  Scedrov,  Wilfried 
Sieg,  Alexei  Stolboushkin,  Alasdair  Urquhart,  Dirk  Van  Gucht,  Moshe  Vardi, 
and  Victor  Vianu,  Other  participants  who  graciously  agreed  to  chair  sessions 
were  Serge  Abiteboul,  Paris  Kanellakis,  Julia  Knight,  Larry  Moss,  Jim  Royer, 
and  Jouko  Vaananen. 

The  papers  in  this  volume  are  grouped  by  themes,  but  the  frequent  difficulty 
of  classifying  papers  demonstrates  in  itself  the  close  relations  between  various 
approaches.  While  contributions  were  not  refereed,  an  editorial  effort  has  been 
made  to  provide  authors  with  informal  feedback  from  readers,  which  in  many 
cases  proved  indeed  to  be  very  constructive.  With  apologies  to  the  many  whose 
help  has  not  been  properly  recorded,  I  would  like  to  thank  for  their  feedback 
Stephen  Bellantoni,  Stephen  Bloch,  Sam  Buss,  Peter  Clote,  Anuj  Dawar,  Georg 
Gottlob,  Hagen  Huwig,  Paris  Kanellakis,  Max  Kelly,  Phokion  Kolaitis,  Julia 
Knight,  Janos  Makowsky,  Anil  Nerode,  Helmut  Schwichtenberg,  Philip  Scott, 
Robert  Seely,  Alasdair  Urquhart,  Moshe  Vardi,  and  Victor  Vianu. 

LCC  was  organized  with  help  from  my  Indiana  University  colleague  and 
friend  Larry  Moss,  to  whom  I  am  most  grateful.  The  meeting  could  not  have 
taken  place  without  generous  grants  from  the  Office  of  Naval  Research,  the 
National  Science  Foundation,  the  Indiana  University  Interdisciplinary  Venture 
Fund,  and  the  Dean  of  the  Indiana  University  School  of  Arts  and  Sciences. 
Speaking  for  all  participants,  I  am  sincerely  obliged  to  these  supporters,  not 
only  for  their  generosity,  but  also  for  their  show  of  confidence  in  the  significance 
of  this  growing  research  community. 


Daniel  Leivant 
Indiana  University 
May  1995 


Table  of  Contents 


Foundational  Issues 

Strict  Finitism  and  Feasibility  1 

Felice  Cardone 

Logical  Omniscience  22 

Rohit  Parikh 

On  Feasible  Numbers  30 

Vladimir  Yu.  Sazonov 

Applicative  and  Proof  Theoretic  Complexity 

On  Parallel  Hierarchies  and  Rl  52 

Stephen  Bloch 

Program  Extraction  from  Classical  Proofs  77 

Ulrich  Berger  and  Helmut  Schwichtenberg 

Computational  Models  and  Function  Algebras  98 

Peter  Clote 

Expressing  Computational  Complexity  in  Constructive  Type  Theory  131 

Robert  L.  Constable 

Light  Linear  Logic  145 

Jean- Yves  Girard 

Intrinsic  Theories  and  Computational  Complexity  177 

Daniel  Leivant 

Complexity  of  Proofs 

On  Herbrand’s  Theorem  195 

Samuel  R.  Buss 

Some  Consequences  of  Cryptographical  Conjectures  for  5^  and  EF  210 

Jan  Krajicek  and  Pavel  Pudldk 

Frege  Proof  System  and  TiVC*’  221 

Gaisi  Takeuti 


VIII 


Computational  Complexity  of  Functionals 
Characterizing  Parallel  Time  by  Type  2  Recursions 

With  Polynomial  Output  Length  253 

Stephen  J.  Bellantoni 

Type  2  Polynomial  Hierarchies  269 

Anil  Seth 

The  Hierarchy  of  Terminating  Recursive  Programs  over  N  281 

Stanley  S.  Wainer 

Complexity  and  Model  Theory 

Feasibly  Categorical  Models  300 

Douglas  Cenzer  and  Jejfrey  B.  Remmel 

Metafinite  Model  Theory  313 

Erich  Grddel  and  Yuri  Gurevich 

Automatic  Presentations  of  Structures  367 

Bakhadyr  Khoussainov  and  Anil  Nerode 

Finite  Model  Theory 


A  Restricted  Second  Order  Logic  for  Finite  Structures  393 

Anuj  Dawar 

Comparing  the  Power  of  Monadic  NP  Games  414 

Ronald  Fagin 

Linear  Constraint  Query  Languages:  Expressive  Power  and  Complexity  426 
Stephane  Grumhach,  Jianwen  Su,  and  Christophe  Tollu 

A  Constant-Space  Sequential  Model  of  Computation  for 

First-Order  Logic  447 

Steven  Lindell 

Logics  Capturing  Relativized  Complexity  Classes  Uniformly  463 

J.A.  Makowsky  and  Y.B.  Pnueli 

Preservation  Theorems  in  Finite  Model  Theory  480 

Eric  Rosen  and  Scott  Weinstein 

A  Query  Language  for  NC  503 

Dan  Suciu  and  Val  Breazu-Tannen 


Strict  Finitism  and  Feasibility 


Felice  Cardone* 
Universita  di  Milano 


Abstract.  We  examine  the  relation  between  predicative  recurrence  and 
strictly  finitistic  tenets  in  the  philosophy  of  mathematics,  primarily  by  fo¬ 
cusing  on  the  role  of  numerical  notations  in  computing.  After  an  overview 
of  Wittgenstein’s  ideas  on  the  “surveyability”  of  notations,  we  analyze 
a  subtle  form  of  circularity  in  the  usual  justification  of  the  primitive  re¬ 
cursive  definition  of  exponentiation  (Isles  1992),  and  suggest  connections 
with  recent  works  on  predicative  recurrence  (Leivant  1993b,  Bellantoni 
&  Cook  1993). 


A  long-standing  thread  in  the  foundations  of  mathematics  questions  the  ontology 
of  numeric  terms  and  the  absoluteness  of  the  informal  notion  of  finiteness.  This 
paper  is  a  preliminary  exploration  of  some  of  the  ideas  (and  puzzles)  underlying 
these  “strictly  finitistic”  approaches,  especially  in  the  light  of  recent  works  that 
relate  predicative  formalisms  to  computational  complexity  (Nelson  1986,  Buss 
1986,  Leivant  1993a).  One  would  expect  investigations  in  these  areas  to  lead  to 
a  unified  perspective,  thereby  contributing  to  an  assessment  of  the  Karp- Cook 
Thesis  on  identifying  feasible  computability  with  poly-time  (see  (Davis  1982); 
this  identification  goes  back  to  Edmonds  1965). 

Large  finite  collections,  and  the  natural  numbers  that  count  them,  are  the  main 
source  of  perplexity  for  strict  finitists:  their  acceptance  makes  the  ontological 
status  of  constructions  requiring  an  arbitrarily  large  but  finite  number  of  steps 
indistinguishable  from  that  of  the  objects  of  platonistic  mathematics.  We  shall 

*  Address:  Dipartimento  di  Scienze  dell’Informazione,  via  Comehco  39/41,  1-20135 
Milano,  Italy.  E-mail:  feliceOimiucca.csi.unimi.it.  This  is  an  expanded  version 
of  my  talk  for  the  Logic  and  Computational  Complexity  Meeting,  and  is  part  of  a 
larger  project  analyzing  the  confluence  of  mathematical  results  and  philosophical 
arguments  iii  the  analysis  of  feasible  computations  that  will  be  described  in  a  future 
joint  paper  with  Daniel  Leivant,  to  whom  I  am  greatly  indebted  for  stimulating  my 
interest  in  this  field  and  for  prompting  me  to  write  this  preliminary  account.  It  will 
be  apparent  from  the  text  that  his  views,  and  also  those  of  David  Isles,  have  been 
quite  influential  on  the  present  paper  (of  course  they  are  not  at  all  responsible  for 
any  of  my  mistakes).  I  am  grateful  also  to  Stephen  Bellantoni,  Paolo  Boldi,  Gabriele 
LoUi,  Diego  Marconi,  Piergiorgio  Odifreddi  and  Nicoletta  Sabadini  for  criticisms 
and  suggestions,  and  to  Roberta  Mari  for  playing  both  as  Proponent  and  Opponent 
in  the  design  of  the  games  described  in  the  last  section.  Finally,  I  would  like  to 
thank  my  friend.  Miss  Claudia  Bonino,  for  betting  long  ago  that  this  paper  would 
eventually  have  been  written.  The  research  was  supported  by  CNR  Cooperation 
Project  “Linguaggi  Applicativi  e  Dimostrazioni  Costruttive”  and  MURST  40%. 
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make  no  attempt  here  to  work  out  a  unified  account  of  strict  finitism,  and  limit 
ourselves  to  a  brief  overview  of  the  web  of  authors  who  contributed  to  it  in 
some  way.  Early  advances  were  made  by  some  of  the  French  pre-intuitionists, 
especially  Emile  Borel,  whose  position  is  developed  at  some  length  in  (Borel 
1952),  and  independently  by  Gerritt  Mannoury  (1909,  1931).  Their  ideas  were 
the  starting  point  of  the  reflections  of  van  Dantzig,  who  is  cited  by  Esenin- Vol’pin 
(1961, 1970)  together  with  Pr^chet  and  Lusin.^  Bernays  (1935)  was  probably  the 
first  to  realize  that  a  notion  of  practical  possibility,  associated  to  the  conception 
of  mathematics  as  a  human  activity,  might  form  the  basis  of  one  possible  position 
in  the  foundations  of  mathematics.  If  this  is  taken  as  defining  strict  finitism  (as 
(Wang  1958),  (Kreisel  1958)  and  also  (Engeler  1971),  in  his  mathematical  model 
of  strict  finitism,  seem  to  suggest),  then  strict  finitism  is  reduced  to  an  extreme 
form  of  constructivism,  which  rules  out  constructions  that  cannot  be  performed 
in  practice  as  well  as  mathematical  statements  whose  proof  depends  on  such  con¬ 
structions.  Later  on,  the  works  of  van  Dantzig  and  Esenin- Vol’pin  complemented 
this  interpretation  and  explicitly  put  forward  a  position  that  has  provided  much 
of  the  basis  for  further  developments  in  this  area.  These  include  the  work  of  van 
Bendegem  (1985);  the  formalization  and  analysis  of  parts  of  Esenin- Vol’pin’s 
work  carried  out  by  Geiser  (1974)  and  Isles  (1981),  and  the  papers  by  Isles 
(1992,  199?).  A  formal  analysis  of  feasibility  is  described  in  (Parikh  1971),  and 
by  different  means  in  (Simon  1977).  Sazonov  (1992)  builds  on  Parikh’s  work, 
and  also  suggests  an  interesting  connection  with  the  Alternative  Set  Theory  of 
Vopenka  (1979),  which  was  originally  intended  also  to  contribute  to  the  formal¬ 
ization  of  vagueness  (Sochor  1984)  and  explicitly  indicated  by  Vopenka  to  be 
applicable  in  particular  to  the  treatment  of  vague  concepts  employed  by  Esenin- 
Vol’pin  (1961)  in  his  construction  of  non-isomorphic  natural  number  series.  On 
the  purely  philosophical  side,  the  vagueness  of  the  concept  of  feasibility  has  orig¬ 
inated  a  series  of  investigations  that  start  from  Dummett’s  analysis  of  paradoxes 
of  the  Sorites  type  (Dummett  1975),  and  culminates  with  (Wright  1982).  Finally, 
in  a  very  different  philosophical  framework,  the  emphasis  on  the  limitations  of 
mathematical  activity  interpreted  as  a  human  and  (therefore)  symbolic  practice 
is  central  to  Rotman’s  “corporeal  semiotics”  of  mathematics  (Rotman  1993). 

A  completely  independent  strictly  finitistic  picture  of  mathematical  activity  also 
emerges  from  the  later  philosophy  of  Wittgenstein.  Kreisel,  in  his  review  of  the 
Remarks  on  the  Foundations  of  Mathematics ,  pointed  out  the  similarity  between 
some  of  Wittgenstein’s  ideas  on  proofs  and  calculations  and  a  position  which  in¬ 
volves  a  distinction,  ignored  by  Hilbert’s  finitism,  between  constructions  which 
consist  of  a  hnite  number  of  steps  and  those  which  can  actually  be  carried  out, 
or  between  configurations  which  consist  of  a  hnite  number  of  discrete  parts  and 
those  which  can  actually  be  kept  in  mind  (or  surveyed)  (Kreisel  1958).  It  was 
later  argued  by  Dummett  (1959)  that  the  emphasis  on  the  practical  notion  of 
possibility  that,  according  to  Kreisel,  is  involved  in  the  concept  of  surveyabil- 
ity,  seems  to  be  somewhat  misplaced.  What  is  really  essential  in  Wittgenstein’s 
arguments  is  instead  the  ramification  of  proof  techniques  that  is  necessary  for 

^  Lusin’s  ideas  have  also  been  mentioned  by  Rashevskii  (1973). 
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dealing  with  increasingly  larger  combinatorial  configurations,  numbers  in  par¬ 
ticular,  as  opposed  to  configurations  that  can  be  easily  taken  in.  This  position 
necessarily  gives  a  prominent  place  to  the  investigation  of  the  role  played  by 
notations  in  computing,  and  will  be  taken  in  the  present  paper  as  one  main 
consequence  of  the  strict  finitist  attitude.  After  an  overview  of  Wittgenstein’s 
remarks  on  surveyability,  stressing  the  cognitive  aspects  of  notations,  we  shall 
turn  in  a  later  section  to  an  orthogonal  -  though  not  unrelated  -  analysis  of 
notations  pertaining  to  their  operational  behavior  and  therefore  more  amenable 
to  formal  treatment  by  means  of  standard  techniques,  inspired  by  recent  works 
of  Isles  (1992),  Leivant  (1993b)  and  Bellantoni  &  Cook  (1993). 

1  Surveyability 


If  you  have  a  proof-pattern  that  cannot  be  taken  in, 
and  by  a  change  in  notation  you  turn  it  into  one  that  can, 
then  you  are  producing  a  proof,  where  there  was  none  before. 

(Wittgenstein  1956,  II  2) 

The  distinction  between  surveyable  and  unsurveyable  configurations  is  central  in 
Wittgenstein’s  critique  of  Russell’s  purely  logical  reconstruction  of  arithmetic. 
The  logicist  project  does  not  reduce  arithmetic  to  logic  because  the  reliability  of 
a  proof  hinges  on  a  criterion  which  is  external  to  the  calculus  itself: 

Imagine  that  you  had  written  down  a  TormuJa’  a  mile  long,  and  you 
shewed  by  transformation  that  it  was  tautologous  ('if  it  has  not  altered 
meanwhile*,  one  would  have  to  say).  Now  we  count  the  terms  in  the 
brackets  or  we  divide  them  up  and  make  the  expression  into  one  that 
can  be  taken  in,  and  it  comes  out  that  there  are  7566  terms  in  the  first 
pair  of  brackets,  2434  in  the  second,  10000  in  the  third.  Now  have  I 
proved  that  2434  +  7566  =  10000?— That  depends— one  might  say—  on 
whether  you  are  certain  that  the  counting  has  really  yielded  the  number 
of  terms  which  stood  between  the  brackets  in  the  course  of  the  proof. 
(Wittgenstein  1956,  II  7) 

It  is  essential  for  arithmetical  calculations  not  only  the  result  but  also  the  tech¬ 
nique  employed  for  arriving  at  it,  and  by  translating  these  into  logical  proofs 
this  intensional  feature  of  calculations  is  lost.  In  fact, 

[.. .]  could  we  find  out  the  truth  of  the  proposition  7034174  +  6594321  = 
13628495  by  means  of  a  proof  carried  out  in  the  [stroke]  notation? — Is 
there  such  a  proof  of  this  proposition? — The  answer  is:  no.  (ibidem,  II 

3) 

This  happens  in  part  also  because  the  notation  has  changed  the  statement  into 
one  that  is  not  surveyable,  and  there  is  no  certainty  that,  in  performing  the 
proof,  the  marks  on  the  paper  do  not  change  without  our  noticing  it: 


Imagine  someone  giving  us  a  sum  to  do  in  a  stroke-notation,  say  { 1 1 1 1 1 1 1 1 1 
+  II I II II 1 1 II  >  while  we  are  calculating,  amusing  himself  by  removing 

and  adding  strokes  without  our  noticing.  He  would  keep  on  saying:  "but 
the  sum  isn’t  right”,  and  we  would  keep  going  through  it  again,  fooled 
every  time. — Indeed,  strictly  speaking,  we  wouldn’t  have  any  concept  of 
a  criterion  for  the  correctness  of  the  calculation. 

(Wittgenstein  1969,  IV  18,  page  330) 

Wittgenstein’s  remarks  on  the  unsurveyability  of  long  numerical  signs,  their 
failure  to  behave  as  paradigms  (Wittgenstein  1956,  II  7)  or  their  lack  of  a  char¬ 
acteristic  pattern  (ibidem,  II  11)  do  not  entail  that  signs  like  ||||||||||  can  never 
be  used  as  tools  for  counting.  In  stone-age  arithmetic  pebbles  were  used  to  count 
the  animals  in  a  herd  and  recheck  the  count  later.  Such  a  stroke-like  notation 
is  also  effective  in  comparing  the  size  of  two  herds  or,  if  the  above  check  is  the 
only  purpose  of  the  counting,  in  adding  two  such  counts,  but  not  for  many  other 
numerical  operations  in  real  life.  The  point  in  this  example  is  that  what  is  being 
performed  here  is  not  a  calculation  but  rather  an  experiment,  that  is,  something 
that  is  outside  arithmetic: 

[...]  “Proof  must  be  surveyable”:  this  aims  at  drawing  our  attention  to 
the  difference  between  the  concepts  of  ^repeating  a  proof’,  and  ^repeating 
an  experiment’.  To  repeat  a  proof  means,  not  to  reproduce  the  conditions 
under  which  a  particular  result  was  once  obtained,  but  to  repeat  every 
step  and  the  result.  And  although  this  shews  that  proof  is  something 
that  must  be  capable  of  being  reproduced  in  toto  automatically,  still 
every  such  reproduction  must  contain  the  force  of  proof,  which  compels 
acceptance  of  the  result.  (ibidem,  II  47,55) 

By  translating  stroke  numerals  into  decimal  notation  (that  is,  as  one  would 
normally  say,  by  counting  them)  we  sometimes  recognize  miscalculations,  but 
this  makes  the  former  technique  to  depend  on  another  one,  and  the  criteria  for 
establishing  the  identity  of  a  proof  external  to  the  calculus  in  which  the  proof  is 
carried  out.  For  this  reason  it  is  important  to  insist  that 

‘A  mathematical  proof  must  be  perspicuous.’  Only  a  structure  whose 
reproduction  is  an  easy  task  is  called  a  “proof”.  It  must  be  possible  to 
decide  with  certainty  whether  we  really  have  the  same  proof  twice  over, 
or  not.  The  proof  must  be  a  configuration  whose  exact  reproduction  can 
be  certain.  (ibidem,  II  1) 

This  emphasis  on  surveyability  clearly  rests  to  a  great  extent  on  an  understand¬ 
ing  of  mathematics  as  a  human  activity  performed  through  the  manipulation  of 
physical  signs  (say,  e.g.,  as  marks  located  in  the  physical  space  or  embodied  in  a 
limited  memory).  It  is  in  such  a  perspective  that  one  must  be  prepared  to  accept 
to  place  on  this  activity  the  natural  limitations  arising  from  the  psychology  of 
human  cognitive  processes: 

beyond  a  certain  length  we  cannot  distinguish  [unary  numerals]  any  fur¬ 
ther  without  counting  the  strokes,  and  so  without  translating  the  signs 
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into  different  ones.  “1 1 1 1 1 1 1 1 1 1  ”  and  1 1 1 1 1 1 1 1 1 H  ”  cannot  be  distinguished  in  the 
same  sense  as  10  and  11,  and  so  they  aren’t  in  the  same  sense  distinct 
signs.  (Wittgenstein  1969,  IV  18,  page  330) 

There  is  a  striking  similarity  between  these  arguments  and  those  that  are  put 
forward  in  Turing’s  analysis  of  mechanical  computation.  In  particular,  it  is  the 
boundedness  of  our  visual  field  that  prevents  us  from  recognizing  at  one  glance 
the  symbols  157767733443477  and  157767733443477  as  being  the  same  (Turing 
1936,  page  251),  and  Turing’s  analysis  takes  into  account  this  impossibility  by 
requiring  that  the  symbols  used  by  his  devices  be  written  in  limited  portions  of 
the  tape  (the  squares)  and  be  immediately  recognizable  (ibidem;  see  also  Kleene 
1952,  §70):  infinitely  many  symbols,  each  of  them  occupying  at  most  only  a  fixed, 
finite  amount  of  space,  would  be  indistinguishable  and  their  encoding  as  strings 
of  simple  symbols  would  exceed,  for  infinitely  many  of  them,  that  amount.^  It 
is  remarkable,  then,  that  according  to  Wittgenstein  one  can  sometimes  make 
surveyable  a  long  symbolic  configuration  by  complicating  it.  So,  for  example, 

[...]  it  is  essential  to  the  calculus  with  1000000  that  this  number  must 
be  capable  of  resolution  into  a  sum  1  + 1  + 1 . . and  in  order  to  be  certain 
that  we  have  the  right  number  of  units  before  us,  we  can  number  the 
units: 

1  +  1  +  1  +  1...  +  1 
1234  1000000. 

This  notation  would  be  like:  TOO,  000.000, 000  ’  which  also  makes  the 
numeral  surveyable.  (Wittgenstein  1956,  II 16) 

Hence  what  is  surveyable  is  not  the  concrete,  spatial  configuration  consisting, 
say,  of  marks  on  paper,  but  rather  its  pattern,  whose  abstract  character  is  shown 
by  its  independence  of  space,  a  non-additive  behavior  which  reveals  its  nature 
of  a  Gestalt: 

How  can  I  know  that  ||||{|||||  and  ||||||||||  are  the  same  sign?  After  all  it 
is  not  enough  that  they  look  alike.  For  having  roughly  the  same  gestalt 
can’t  be  what  is  to  constitute  the  identity  of  the  signs,  but  just  their 
being  the  same  in  number. 

(The  problem  of  the  distinction  between  l  +  l-fl  +  l  +  lH-l  +  l  and 
H-l-hl  +  l-hl  +  l  +  l  +  lis  much  more  fundamental  than  appears  at 
first  sight.  It  is  a  matter  of  the  distinction  between  physical  and  visual 
number.)  (Wittgenstein  1969,  IV  18,  page  331) 

Observe  that  it  is  unlikely  that  this  interpretation  of  surveyability  may  be  refor¬ 
mulated  in  terms  of  devices  operating  in  a  manner  sufficiently  similar  to  that  of 

^  Especially  interesting  in  the  present  perspective  is  the  formulation  of  the  principles 
of  mechanisms  carried  out  by  Gandy  in  terms  of  an  analogy  with  physics,  and  based 
on  the  existence  of  a  lower  bound  on  the  linear  dimensions  of  every  atomic  part  of 
the  device  and  of  an  upper  bound  (the  velocity  of  light)  on  the  speed  of  propagation 
of  changes  (Gandy  1980,  page  126). 
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Turing  machines.  As  observed  by  Gandy  (1980),  Gestalten  have  a  global  char¬ 
acter  that  is  irreducible  to  the  iteration  of  simple  local  steps  proper  to  such 
mechanisms.  Wittgenstein’s  examples,  instead,  exploit  the  fact  that  this  can 
happen  in  our  use  of  symbolic  configurations  when  performing  calculations  or 
proofs.  This  is  not  in  conflict  with  the  existence  of  limitations  upon  our  ability 
to  recognize  patterns:  it  simply  shifts  the  emphasis  on  these  as  the  true  objects 
of  attention  when  carrying  out  the  symbolic  manipulations  involved  in  our  non 
formalized  proofs.  Also,  it  is  in  this  interpretation  that  the  geometric  properties 
of  proofs  are  of  a  special  relevance: 

[■  •  •]  logical  proof,  e.g.  of  the  Russellian  kind,  is  cogent  only  so  long  as  it 
also  possesses  geometrical  cogency.  And  an  abbreviation  of  such  a  logical 
proof  may  have  this  cogency  and  so  be  a  proof,  when  the  Russellian 
construction,  completely  carried  out,  is  not. 

[...]  The  consideration  of  long  unsurveyable  logical  proofs  is  only 
a  means  of  shewing  how  this  technique — which  is  based  on  the  geom¬ 
etry  of  proving— may  collapse,  and  new  techniques  become  necessary. 
(Wittgenstein  1956,  II  43,45) 

Surveyability  is  just  the  aspect  through  which  the  geometrical  cogency  of  proofs 
becomes  manifest;  admittedly,  this  is  a  vague  property  of  symbolic  configura¬ 
tions,  and  a  common  objection  to  according  any  relevance  to  it  is  that  seeing 
whether  [a  proof  of  a  Russellian  proposition  stating  an  addition  like  ^a  +  b  =  c\ 
consisting  of  a  few  thousand  signs]  is  correct  or  not  is  a  purely  external  dif¬ 
ficulty,  of  no  mathematical  interest.  (“One  man  takes  in  easily  what  someone 
else  takes  in  with  difRculty  or  not  at  all”  etc.  etc.)  but  this  objection  implicitly 
presupposes  that  the  definitions  serve  merely  to  abbreviate  the  expression  for 
the  convenience  of  the  calculator;  whereas  they  are  part  of  the  calculation.  By 
their  aid  expressions  are  produced  which  could  not  have  been  produced  with¬ 
out  it  (ibidem,  II  2).  It  appears  at  this  point  that  the  stress  on  surveyability, 
rather  than  being  a  study  of  practical  limitations,  takes  these  for  granted  in 
order  to  develop  a  theory  of  meaning  for  mathematical  statements  close  in  spirit 
to  the  theory  of  meaning  underlying  intuitionistic  mathematics,  as  advocated 
by  Dummett  (1977).  This  was  pointed  out  by  Dummett  himself  in  relation  to 
Wittgenstein’s  ideas:  what  the  word  ‘prime^  means  as  applied  to  large  numbers 
is  shown  by  what  we  accept  as  the  criterion  for  primality,  what  we  take  as  the 
standard  whereby  to  assess  claims  that  a  number  is  prime  or  composite  (Dum¬ 
mett  1959,  page  181).  Notations  do  have  a  role  in  this  perspective  because  their 
being  surveyable  or  not  may  necessitate  a  change  in  the  techniques  for  manipu¬ 
lating  them,  and  it  is  by  means  of  techniques  that  we  use  signs,  and  give  them 
their  meaning.  Strict  finitism  is  not  a  restriction  of  the  acceptable  constructions 
to  those  that  are  feasible,  in  particular  strictly  finitistic  arithmetic  is  not  that  of 
feasible  numbers,  but  a  controlled  use  of  notations  that  takes  into  account  the 
fact  that  it  is  not  always  possible  to  pass  from  one  notation  to  another  without 
changing  the  method  of  counting,  and  thereby  the  meaning  of  the  statement  in 
which  the  notation  occurs.  This  means  in  particular  that  it  is  legitimate  from 
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the  point  of  view  of  (this  interpretation  of)  strict  finitism  to  use,  for  example, 
exponential  notations: 

one  does  want  to  use  numbers  like  6.0229  x  10^®  (Avogadro’s  number), 
even  if  we  cannot  count  up  to  it.  We  can  write  it  down,  add  10^^  to  it 
and  write  that  down  -  in  other  words  we  can  manipulate  it. 

(Simon  1977,  page  195) 

2  Notations  and  numbers 

What  are  numbers?  -  What  numerals  signify;  an  investigation  of 
what  they  signify  is  an  investigation  of  the  grammar  of  numerals. 

(Wittgenstein  1969,  IV  18,  page  321) 

Making  explicit  when  two  notations  (or  two  techniques)  are  equivalent  is  a  major 
problem  arising  from  Wittgenstein’s  remarks,  as  pointed  out  already  by  Kreisel 
(1958).  An  especially  interesting  instance  of  this  problem  appears  in  an  inten- 
sional  counterpart  of  the  totality  proofs  of  recursive  functions,  that  in  a  strictly 
finitistic  context  arises  already  in  the  case  of  functions  defined  by  primitive 
recursion,  notably  exponentiation  (Esenin- Vol’pin  1970),  (van  Dantzig  1956). 
Sometimes,  when  introducing  a  new  notation  by  means  of  such  a  definition,  we 
are  more  likely  to  introduce  another  way  of  counting  that  must  be  kept  dis¬ 
tinct  from  the  counting  by  units  implicit  in  the  conception  of  numerals  as  the 
expressions  obtained  from  0  by  application  of  s(*). 

The  problematic  status  of  primitive  recursive  definitions  can  be  seen  already 
from  the  fact  that  the  usual  set- theoretical  justifications  of  primitive  recursion 
follow  the  same  line  of  the  original  one  of  Dedekind  (1901)  in  using  impredica- 
tive  comprehension  principles  (see  also  (Henkin  I960)),  and  there  seems  to  be 
an  irreducible  amount  of  impredicativity  in  any  attempt  to  avoid  introducing 
natural  number  series  as  processes  (see  (Parsons  1992)  for  a  discussion  of  this 
aspect  of  the  induction  principle).  Natural  means  for  analyzing  the  ideas  behind 
a  genetic  view  of  numbers,  still  preserving  to  a  limited  extent  the  possibility  of 
inductive  reasoning,  lead  directly  to  a  logical  characterizations  of  the  class  of 
functions  computable  in  polynomial  time  in  (Leivant  1993a),  and  provide  a  link 
between  the  theory  of  feasible  computations  and  the  foundational  issues  that  are 
in  the  background  of  most  strict  finitist  positions.  This  analysis  also  yields,  as 
an  important  result,  a  notion  of  tiering  which  allows  the  description  of  feasibly 
computable  functions  by  means  of  equational  systems  in  which  tiers  are  used  as 
sorts  that  prevent  the  formation  of  (primitive  recursive)  functions  which  violate 
predicatively  justifiable  constraints  (Bellantoni  1992,  Bellantoni  &  Cook  1993 
and  Leivant  1993b). 

Independently,  an  argument  aimed  at  showing  the  existence  of  a  subtle  form 
of  circularity  in  the  ordinary  proofs  of  totality  of  exponentiation  has  been  re¬ 
cently  developed  by  Isles  (1992, 199?)  in  the  framework  of  first-order  theories,  in 
particular  Peano  arithmetic  (see  also  (Esenin- Vol’pin  1970,  page  33)  for  a  related 
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claim).  By  dropping  the  assumption  that  variables  range  over  the  whole  model 
of  the  theory,  it  becomes  possible  to  set  up  a  system  for  annotating  the  inference 
within  a  proof  by  means  of  “arrows”  which  describe  the  referential  assumptions 
on  the  variables  implicit  in  the  justification  of  the  correctness  of  each  applica¬ 
tion  of  the  rule.  This  analysis  is  intensional,  in  that  different  proofs  of  the  same 
formula  may  lead  to  different  assumptions  on  the  range  of  the  same  variables 
(“reference  grammars”  in  the  terminology  of  (Isles  199?)).  It  turns  out  that  the 
usual  inductive  proofs  of  totality  of  exponentiation  require  for  their  correctness 
that  the  underlying  model  be  already  closed  under  the  exponentiation  function. 
Both  of  these  approaches  emphasize,  though  in  different  forms,  two  symbolic 
levels  involved  in  definitions  of  functions:  on  one  hand  we  deal  with  basic  ex¬ 
pressions,  whose  numerical  meaning  is  unproblematic;  on  the  other  hand,  we 
introduce  new  notations  whose  reducibility  to  those  of  the  first  level  has  to  be 
proved  in  some  way,  under  specific  assumptions. 

Beside  these,  we  propose  below  a  natural  operational  analysis  of  primitive  recur¬ 
sion  that  contributes  to  put  in  a  formal  setting  at  least  some  of  the  intensional 
aspects  involved  in  transforming  one  notation  into  another,  and  also  allows  to 
single  out  some  necessary  conditions  for  the  possibility  of  such  transformations 
that  we  shall  outline  in  some  cases  related  to  the  usual  primitive  recursive  algo¬ 
rithm  for  exponentiation,  corresponding  to  those  discussed  in  (Isles  1992).  At  the 
intensional  level  the  totality  of  a  function  corresponds  to  the  possibility  of  elimi¬ 
nating  the  defined  symbol  denoting  it.  Any  expression  in  which  a  defined  function 
symbol  occurs  gives  rise  to  a  reduction  process  directed  to  the  elimination  of  it 
and  of  any  other  defined  symbol  contributing  to  its  definition,  hereditarily.  Ob¬ 
serving  that  primitive  recursive  definitions  are  instances  of  the  orthogonal  (= 
left-linear  and  without  critical  pairs)  term  rewriting  systems  (Klop  1990),  we 
apply  to  our  examples  some  techniques  introduced  in  (Huet  h  Levy  1979)  for 
analyzing  computations  in  this  class  of  systems.  The  main  goal  of  our  proposal, 
not  achieved  at  present,  will  be  to  develop  semantical  counterparts  of  tiering 
that  may  suggest  an  alternative  explanation  of  the  circularities  brought  to  light 
by  Isles’  analysis. 

2.1  Redexes  and  derivations 

We  recall  the  basic  notions  needed  for  our  analysis:  most  of  them  are  standard 
concepts  from  the  theory  of  term  rewriting  systems,  and  we  refer  to  (Huet  h 
Levy  1979)  and  (Klop  1990)  for  a  thorough  development.  We  shall  consider 
terms  built  over  constructors  0  and  s(-),  using  variables  from  a  set  V  and 
symbols  for  defined  n-ary  functions  (n  >  0).  Primitive  recursion  takes  then  the 
form: 

F(x,0)  =  M[f] 

F(f,s{3/))  =  Ar[J,y,F(f,2/)] 

where  M,  N  are  terms  possibly  containing  occurrences  of  the  subterms  listed 
within  square  brackets.  Primitive  recursive  definitions  can  thus  be  seen  as  exam¬ 
ples  of  term  rewriting  system  with  constructors,  in  which  there  is  a  partition  of 
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function  symbols  into  defined  symbols  V  and  constructors  C  and  each  rewriting 
rule  has  the  form  . . . ,M„)  =  iV,  where  F  £  V  and  the  only  function 

symbols  occurring  in  each  Mj,  for  i  =  1, . . .  ,n,  are  in  C. 

For  many  purposes  it  will  be  necessary  to  keep  track  of  the  position  of  a  symbol, 
and  this  is  done  by  means  of  “Dewey’s  decimal  notation”  (see  Knuth  1968,  pages 
310  ff.):  the  position  (sometimes  also  called  occurrence)  of  a  symbol  in  a  term 
M  is  given  by  the  sequence  of  (positive  integers  assigned  from  left  to  right  to  the) 
nodes  along  the  unique  path  from  the  root  to  that  symbol  in  the  construction 
tree  of  the  term,  the  root  position  being  the  empty  sequence  €.  These  form  the 
set  (9(M),  which  can  be  ordered  by  the  relation  u  ^  v  which  holds  only  if 
u,v  e  0(M)  and  «  is  a  prefix  of  -y.  If  two  positions  it,  v  are  incomparable  in  this 
ordering  they  are  said  to  be  disjoint,  written  u\v;ifu-<v  then  their  difference 
is  i;  —  It,  the  position  which  consists  of  the  symbols  of  v  except  for  the  initial 
segment  it.  The  subterm  of  M  whose  principal  symbol  is  at  position  it  is  denoted 
by  M/it:  its  positions  are  all  the  differences  i;  —  it  for  i;  €  0{M)  such  that  v  y  u. 
The  root  symbol  of  M/it  is  then  M(it),  looking  at  a  term  as  a  (partial)  function 
from  positions  to  symbols.  If  a  term  M  contains  a  subterm  P  at  some  unspecified 
position  It,  we  can  consider  a  new  term  M'[  ]  which  is  like  M  except  for  having 
a  special  symbol  [  ],  a  hole,  at  position  u.  M'[  ]  is  called  a  context,  and  M 
can  be  regarded  as  the  term  M'[P],  in  which  the  hole  has  been  filled  with  P. 

A  redex  is  any  term  R  for  which  there  exists  a  substitution  a  of  terms  for 
variables  such  that  R  =  cr(A),  where  A  =  p  is  one  of  the  rules  of  the  system;  A  is 
said  in  this  case  to  be  the  redex  scheme.  Given  a  term  M  =  ATfP]  containing 
such  a  redex  as  a  subterm,  an  elementary  derivation,  written  M 
replaces  R  =  a-(A)  with  its  contractum  R!  =  «^(p),  yielding  M*  =  iV[P']. 
If  M/u  is  the  redex  P,  then  the  elementary  derivation  contracting  u  is  written 
M  Af'.  By  refiexive  transitive  closure  we  obtain  a  reduction  relation  P  — Q 
which  holds  whenever  there  is  a  derivation  starting  at  P  and  ending  at  Q,  i.e. 
a  sequence  of  terms  Mo, . . . ,  Mn  such  that 

P  —  Mq  —►tto  Afl  •  •  •  “►tin-2  Afji— 1  Afyi  =  Q. 

with  each  ui  belonging  to  the  set  1Z{Mi)  of  redex  occurrences  in  the  term  Mj. 
A  term  containing  no  redex  as  a  subterm  is  said  to  be  in  normal  form.  The 
preceding  definitions  can  be  generalized  to  multiderivations  that  at  each  step 
contract  finitely  many  disjoint  redex  occurrences.  The  notation  M  N  indi¬ 
cates  the  existence  of  a  multiderivation  with  initial  term  M  and  final  term  N, 
composed  of  elementary  multiderivations 

M  =  Mo  ^Uo  Afl  =>Ui  M2  =>U2  '  *  •  ^Un-2  Mn-i  =>Un  Mn  =  AT, 


where  Ui  C  P(Mi)  for  i  <  n.  Clearly,  derivations  can  be  considered  as  multi¬ 
derivation  which  at  each  step  contract  a  singleton. 
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2.2  Residuals 

It  is  important  to  have  some  way  to  keep  track  of  the  amount  of  work  performed 
by  a  multiderivation  A  :  M  P  with  respect  to  another  one  B  :  M  Q 
starting  at  the  same  term  (in  this  situation  A  and  B  are  said  to  be  coinitial).  The 
residual  of  A  after  B,  in  symbols  A\B,  is  the  multiderivation  A\B  :  Q  =^*  N 
that,  intuitively,  performs  the  steps  of  A  that  are  left  after  performing  B.  Then  A 
and  B  are  permutation  equivalent,  in  symbols  A  =  B,  whenever  (7\i4  =  C\B 
for  any  multiderivation  C  coinitial  with  both.  By  defining  the  join  AuB  of  A  and 
B  as  the  concatenation  A\  {B\A),  we  can  thus  say  that  B  performs  more  work 
than  A  if  B  ~  AU B.  The  Parallel  Moves  Lemma,  a  strong  form  of  the  Church- 
Rosser  property,  states  that  AuB  =  BuA^foi  coinitial  multiderivations  A, B. 
These  ideas  were  introduced,  for  orthogonal  term  rewriting  systems,  by  Huet  and 
Levy  (1979)  who  also  developed  the  underlying  theory:  we  summarize  below  the 
basic  definitions  and  facts  that  we  shall  need,  referring  to  this  classic  for  the  full 
details. 

Given  an  elementary  derivation  a  :  M  — N  using  the  rewriting  rule  A  =  p,  and 
an  arbitrary  v  €  0(M),  the  set  v\a  is  the  set  of  positions  in  0{N)  defined  as: 

{{v}  if  i;  I  w  or  v  -<  u 

{uw'v^  I  p/w'  is  a  variable  x}  if  v  =  uwv^  and  X/w  ~  x 
0  otherwise 

Observe  that,  ii  w  e  v  —  u,  then  M{w)  =  N(w).  For  a  derivation  a;  6  obtained 
by  concatenating  the  elementary  derivation  a  :  M  ^  M’  with  the  derivation 
&  :  M'  — ►*  i\r,  and  any  v  e  0(M)y  define 

v\a;  b  =  |  w  €  v\a}. 

When  A  :  M  N  is  Sin  elementary  multiderivation  with  U  =  {ui, . . . ,  and 
V  e  0{M),  define  u\A  =  v\(a^(i); . . .  ;a,r(n)),  where  tt  :  {1, . . .  ,n}  {1, . . .  ,n} 

is  any  permutation  and  aj  is  the  elementary  derivation  contracting  uj.  Then,  for 
any  multiderivation  A  and  v  €  0{M)  define  v\A  by  induction  of  the  length  \A\ 
of  A  as  follows,  where  id  is  the  empty  multiderivation: 

{v\  id  =  {v} 
v\A;B  =  \J{w\B\w€v\A}, 

If  U  C  0{M)  consists  of  disjoint  positions,  then 

U\A  =  lj{w\^  \ueU}. 

Finally,  for  coinitial  multiderivations  A,  B  with  B  :  M  Ny  set 

B\A  =  U\Ay 

and  an  application  of  the  Parallel  Moves  Lemma  justifies  the  extension  of  the 
residual  relation  to  multiderivations  of  arbitrary  length: 
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"  For  coinitial  A,  B,  with  B  elementary  multiderivation: 

/id\B  =id 

\  {Ai-,A2)\b  =  (aab);(^2\(b\Ai)). 

—  For  arbitrary  coinitial  multiderivations  A,  B: 

/  A\  id  =  A 
\A\Bi;B2  =  {A\Bi)\B2. 

2.3  Outside-in  reductions 

In  order  to  analyze  the  computational  aspects  of  primitive  recursive  definitions 
relevant  to  the  present  discussion,  we  need  to  be  able  to  put  any  derivation  into 
a  canonical,  permutation  equivalent  form  (Huet  &  Levy  1979).  This  generalizes 
the  Standardization  Theorem  of  the  A-calculus  to  orthogonal  term  rewriting 
systems,  where  the  choice  of  the  leftmost-outermost  redex  at  each  step  does  not 
lead  in  general  to  standard  reductions. 

For  a  redex  occurrence  u  G  lt{M)  with  redex  scheme  A,  we  can  define  the  set  of 
positions  that  form  the  pattern  of  the  redex  at  w  as: 

Cm{^)  =  {v  6  0{M)  \  u-<v  and  A/v  ^  V}. 

Given  any  position  u  G  (9(M),  say  that  a  multiderivation  A  :  M  =>*  N  pre¬ 
serves  u  if  at  each  step  of  A  no  redex  occurrence  u  >-  «  is  contracted. 

Definition!  (Huet  &  Levy  1979,  page  11).  A  position  u  G  0{M)  is  ex¬ 
ternal  for  a  multiderivation  A  :  M  =^*  AT,  written  u  G  A'(A),  when 

“  either  A  preserves  u,  or 

-  A  can  be  decomposed  into  Ai;  A2;A3  such  that: 

•  Ai  preserves  u, 

•  A2  :  P  =^v  Q,v  eV  and  u  G  Cp(v),  for  some  (unique)  v  -<  w, 

•  G  A'(A3). 

Given  a  multiderivation 

A  :  Mo  =>Ui  Ml  =^U2  ^2  Mn 

and  u  G  7^(Mo),  we  say  that  u  is  an  initial  redex  occurrence  pertaining  to  A, 
written  u  G  P.(A),  if  Ui  fl  u\A[2  -  1]  ^  0  for  some  i  <  n,  where 

A[i  -  1]  ^  Mo  =^f/i  Ml  ^U2  ^2  •  *  •  =>Ui-i  Mi_i. 

The  set  of  external  redex  occurrences  of  A  is  defined  as  the  set 

e{A)  =  7^(A)  n  Ar(A). 


A  derivation  A  is  outside-in  if 
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“  either  A  =  id,  or 

-  A  —  Ai;A2,  for  some  elementary  derivation  Ai  contracting  u  €  £{A)  and 
A2  outside-in. 

The  main  property  of  outside-in  derivations  is  their  ubiquity  (Huet  &:  Levy 
1979,  page  15):  every  multiderivation  A  caii  be  transformed  into  an  outside-in 
derivation  B  such  that  A  =  B. 

2.4  Computational  behavior  of  primitive  recursive  definitions 

Assume  that  a  term  M  contains  an  occurrence  u  of  a  symbol  defined  by  means 
of  primitive  recursion.  For  a  derivation  a:  M  N  contracting  a  redex  M/v  at 
V  by  applying  the  rule  A  =  p,  define  the  relation 

u  >a'i^  (u  causes  w) 

for  w  e  0{N)  by  the  clauses: 

-  if  u  =  V,  then  u  t> a 'UJ  for  all  positions  w  G  0{N)  such  that  w  =  wit;', 
w'  G  0{p)  and  p(w')  is  a  defined  symbol; 

-  otherwise,  u  >«  u;  for  all  ly  G  u\a. 

Extend  this  notion  of  causality  to  elementary  multiderivations  A  :  M  =>ir  iV, 
by  defining,  for  u  G  0{M)  and  w  G  0{N),  u  >aw  if  and  only  if  w  u;  for 
some  elementary  derivation  a  contracting  v  e  U:  this  definition  is  justified  by 
the  disjoint  ness  of  redexes  in  U.  Finally,  for  an  arbitrary  multiderivation 

A  =  Ai;A2  :  M  =>Ui  Mi  M2  =>Ua  ' '  *  ^ 

such  that  Ai  :  M  Mi,  A2  :  Mi  =>*  AT,  with  u  G  0{M)  and  w  G  0{N): 

u  ly  if  and  only  if  there  is  such  that  u  >Ai  u'  and  u'  t>A2 

The  descendants  of  the  occurrence  u  of  a  defined  symbol  in  M  through  a 
multiderivation  A  :  M  =>►*  iV  are  the  elements  of  the  set  Aa{u)  =  {ly  G  0{N)  \ 
u  >A'^}‘  A  eliminates  the  occurrence  u  G  0{M)  of  the  defined  symbol  if 
AA{y)  =  0.  Clearly,  this  notion  is  invariant  under  permutation  equivalence. 

A  multiderivation 

A  :  M  =  Mo  ^Uo  Ml  *  *  •  ^Um-i  Mm  =  N 
is  said  to  contain  a  multiderivation 

B  :  Nq  =>Vo  =>Vi  •  •  •  =^Vrr^-l 

if  each  Ni  is  a  subterm  of  Mi  and  the  redexes  in  Vi,  for  Vi  #  0,  have  the  form 
UiV,  where  Ni  =  Mi/ui  and  v  G  7l(iVi).  A  multiderivation  A  :  M  N,  issued 
from  a  term  of  the  form  M  =  M'[P],  is  said  to  count  up  to  P  if  A  contains 
multiderivations 


Pi  :  P  s(Pi),  P2  :  Pi  s(P2), . . . ,  Pn-i  :  Pn-i  s(P„),  Bn  :  P„  0. 
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These  give  a  multiderivation  B  :  P  n,  where 

n=  sOj^(O)---), 

n  times 

for  some  n  6  CJ.  This  notion  is  not  in  general  preserved  under  permutation 
equivalence  of  derivations  (consider  for  example  the  two  equivalent  derivations 
from  the  term  K{Qj  C)  using  the  rules  K{x^y)  =  x  and  C  =  0),  but  we  can  still 
prove  the  following:  , 

Lemma  2.  If  a  :  M[P]  — ►*  N  is  an  outside-in  derivation  that  counts  up  to  P, 
then  any  B  =  a  counts  up  to  P.  □ 

Consider  now  the  primitive  recursive  definition  of  addition: 

+(x,0)  =  x 

+{x,s{y))  =  s(+{x,y)). 

We  have  the  following  property: 

Propositions.  Any  derivation  A  starting  at  +(P, Q)  which  eliminates  the  root 
occurrence  of  the  addition  symbol  counts  up  to  Q.  . 

Proof:  We  can  assume  without  loss  of  generality  that 

A:  Mo -Ml-. - >Mn 

is  already  outside-in,  and  use  induction  on  its  length:  the  conclusion  then  follows 
by  the  preceding  Lemma.  If  the  position  e  G  0(+(P,  Q))  is  not  already  a  redex, 
it  will  eventually  become  one  because  A  eliminates  e  and  therefore  AAi^)  =  0, 
which  implies  that  e\A  =  0.  Hence  A  =  Ai;A2]A2  with  Ai  :  Mq  — *  Mi, 
A2  :  Mi  —£  Mi+i  and  A^  :  Mi+i  — *  M„.  As  A  is  outside-in,  all  contractions 
in  Ai  take  place  below  position  2.  In  fact,  assume  that  some  position  lu  is 
contracted  in  Ai  at  step,  say,  Mk’.  then  this  position  has  to  be  external  for  the 
derivation 

A[k,  n] 

and,  not  being  preserved  by  A,  this  position  must  be  necessary  to  create  the 
redex  at  position  e  in  Mi,  which  is  impossible  as  CMi{€)  =  {€,2}.  Hence  A2 
preserves  2,  and  this  derivation  is  such  that 

1.  either  Ai  :  Q  — *  s(Q'),  or 

2.  Ai  :  Q  0. 

In  the  second  case,  the  root  symbol  is  eliminated  after  performing  Ai;  A2  and  A 
counts  up  to  Q.  In  the  first  case,  we  have: 

Ai:+(P,Q)^*+(P,s(Q')), 

A2:+(P,s(0'))  s(+(P,Q')), 
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and  €  is  external  for  ^3.  Actually,  in  this  case  the  position  1  is  preserved  by  A3, 
and  we  can  consider  an  induced  derivation: 

eliminating  the  root  symbol.  A!  is  outside-in  and  |A'|  <  |A|,  hence  A!  counts  up 
to  Q'  by  induction  hypothesis,  so  that  there  is  a  derivation  Q  ->*  s(”)(0),  for 
some  Then  by  composition  we  obtain 

Q  -V*  s(Q')  s("+i)(0) 

showing  that  A  counts  up  to  Q.  □ 

A  corresponding  property  can  be  shown  to  hold  also  for  the  primitive  recursive 
definition  of  multiplication: 


*(a;,0)  =  0 

*{x,s{y))  =  +{*{x,y),x). 


In  this  case  we  have: 

Proposition 4.  Any  derivation  A  starting  at  *{P,Q)  which  eliminates  the  root 
occurrence  of  the  multiplication  symbol  counts  up  to  Q  and,  if  Q  — s(Q')  for 
some  Q',  then  A  counts  also  up  to  P. 

Proof:  Assume,  as  before,  that  A  is  outside-in,  and  observe  that  e  must  even¬ 
tually  become  a  redex  occurrence  contracted  in  the  derivation  A[i,n],  where 

A  :  *{P,Q)  =  Mo-^Mi^ - ^ 

Hence  A  can  be  decomposed  into  Ai;A2;Az,  where 

Ai:*{P,Q)  Mi 
A2  :  Mi  Afi+i 
A3  :  Mt+i  Mn. 

As  A  is  outside-in  each  redex  contracted  in  Ai,  say  at  step  k  <  n,  must  be 
external  for  A[k,n],  hence  they  must  be  below  the  position  2  (in  fact,  by  defini¬ 
tion,  the  positions  below  1  are  not  necessary  for  creating  the  redex  at  the  root 
position  in  Mi).  Then  Ai  preserves  2,  and  we  have  that 

1.  either  Q  -4*  s(Q'), 

2.  or  Q  0. 

In  the  second  case  Ai  counts  up  to  Q  (but  not  up  to  P).  In  the  first  case,  after 
the  contraction  A2  of  the  root  redex,  we  have 

As:-\-{*{P,Q^),P)  ^*M„ 


15 


eliminating  the  descendants  of  e  in  Ai\A2.  In  particular,  this  entails  that  Az 
counts  up  to  P  by  the  preceding  Proposition,  hence  A  counts  up  to  P.  If  P  -4* 
s(”^)(0)  for  some  n  G  oj,  then  we  can  further  decompose  Az  into 

M„, 

where  A3  is  the  shortest  initial  segment  of  A3  eliminating  the  root  position  of 
We  also  know  that  no  redex  inside  is  contracted  by  A3, 

hence 

and  Ag  induces  a  derivation 

A' :*{P,Q')^*Mn 

that  counts  up  to  Q'  by  induction  hypothesis,  as  |A'|  <  |A|.  Finally,  we  have 
that  A  counts  up  to  Q  by  suitably  composing  Ai  and  the  derivation  showing 
that  A'  counts  up  to  Q'.  □ 

From  this  property  it  follows  also  that  a  derivation  eliminating  the  root  occur¬ 
rence  of  a  multiplication  symbol  from  *{P,Q)  reduces  that  term  to  normal  form, 
unlike  what  happens  with  addition. 

Corollary  5.  If  A  starts  at  *(P,  Q)  and  eliminates  the  root  occurrence  of  the 
multiplication  symbol,  let  Q  m  and,  if  m  >  0,  let  P  -**  n.  Then  A  counts 
up  to  p,  where  p  —  m*n.  □ 

If  we  consider  now  the  primitive  recursive  definition  of  exponentiation: 

T  (a:,0)  =  s(0) 

T(a:,s(y))  =  *(T  ix,y),x). 

we  see  that,  as  a  consequence  of  the  preceding  properties,  in  order  to  eliminate 
the  root  occurrence  of  the  exponentiation  symbol  in  |  (P,0)  it  is  necessary 
to  count  up  to  Q  and,  if  Q  ->*  s(Q')  for  some  Q',  also  up  to  t  {PiQ')  (and, 
by  the  preceding  Corollary,  up  to  t  (PjQ))-  There  is,  after  all,  a  difference  in 
the  primitive  recursive  definitions  of  addition,  multiplication  and  exponentiation 
that  shows  up  in  the  form  of  the  necessary  conditions  that  the  arguments  and 
results  of  these  functions  have  to  satisfy  in  order  to  make  possible  the  elimination 
of  top  level  occurrences  of  the  defined  symbol.  While  in  the  cases  of  addition  and 
multiplication  we  need  only  assume  that  some  or  all  of  the  inputs  are  standard, 
the  primitive  recursive  definition  of  exponentiation  makes  necessary  to  introduce 
the  assumption  that  the  (previous  value  of  the)  output  is  standard.  This  is  one 
example  in  which  a  notation  which  is  being  introduced  by  a  definition  is  assumed 
to  be  interchangeable  with  a  primitive  one,  although  this  conclusion  is  not  as 
strong  as  the  kind  of  circularity  revealed  by  the  analysis  carried  out  in  (Isles 
1992),  because  of  the  logic-free  character  of  the  equational  formalism.  Rather, 
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the  above  results  seem  to  enable  a  comparison  with  the  typing  of  these  functions 
in  the  2-tiered  version  of  the  formalism  proposed  in  (Leivant  1993b),  having 
constructors  0  and  s(*)  and  tiers  No,Ni,  where  a  primitive  recursive  function 
defined  by: 


f{x,0)=9o(x) 

/(^,  s(2/))  =  9s  {f{x,  2/),  2/,  x) 

has  tier  A/*  x  ►  Nj  provided  ^Oj^s  have  respectively  tiers  J\f  Nj  and 
Nj  X  Ni  X  A/*  Nj  for  i  >  j,  where  Nf,  Nj  G  {No,  Ni}  and  AA  is  a  product  of 
tiers  of  length  |x|.  In  fact  we  have  that  4- :  No  xNi  No  and  *  :  Ni  xNi  — ^  No, 
while  exponentiation  cannot  be  typed  at  all.  It  is  tempting  therefore  to  assign 
tier  Ni  to  the  arguments  that  will  eventually  be  reduced  to  normal  form  (a 
unary  numeral)  in  any  computation  eliminating  the  defined  symbol. 

2.5  A  playful  interpretation 

Is  there  a  formal  sense  in  which  the  peculiar  computational  behavior  of  expo¬ 
nentiation  involves  some  kind  of  circularity? 

We  imagine  a  Proponent  of  a  primitive  recursive  definition  who  is  committed 
to  provide  his  Opponent  with  evidence  that  it  can  be  eliminated.  The  scenario 
takes  then  the  form  of  a  dialogue  between  the  Proponent  and  the  Opponent,  in 
the  tradition  of  Lorenzen  (1961)  (see  also  (Stegmuller  1964)  and  the  more  recent 
(Felscher  1986)).  We  shall  use  these  dialogical  games  as  an  heuristic  device  for 
understanding  in  a  different  way  the  assumptions  behind  the  use  of  primitive 
recursive  definitions  in  the  above  examples.  Formal  rules  for  them  may  be  de¬ 
signed  along  the  same  lines  as  those  for  the  dialogical  games  recently  proposed 
in  (Abramsky,  Jagadeesan  and  Malacaria  1994)  and  (Hyland  &  Ong  1994)  in 
their  constructions  of  fully  abstract  models  for  typed  A-calculus  with  recursion, 
and  from  those  for  algorithms  on  sequential  data  structures  described  in  (Curien 
1992).  (Actually,  we  shall  mix  the  description  of  a  typical  play  of  the  game  with 
that  of  a  “meta-game”  between  Proponent  and  Opponent,  whose  aim  is  to  choose 
the  rules  in  such  a  way  that  they  are  unbiased  toward  one  of  the  players.) 

A  game  is  uniquely  associated  with  a  primitive  recursive  definition  of  the  form: 

F(Xi,...,Xn,0)  =  M[Xi,,,.,Xn] 

F{xi , . . . ,  Xn,  s{y))  =  N[xi , . . . ,  2/,  F{xi , . . . ,  ,  2/)] 

The  positions  are  built  by  coloring  the  symbols  occurring  in  primitive  recursive 
terms  by  O  or  P:  a  player  can  only  make  moves  relative  to  symbols  of  his  color. 
Positions  consist  of  cells  or  values  where,  for  i  G  {0,P},  ^-cells  are  filled  by 
^-values: 


cell  ::=  0-cell  |  P-cell 
0-cell  ::=  s(O-cell)  |  <^P(cell, . . . ,  cell) 
P-cell  ::=  s(P-cell)  |  ^x  \  timed  P-cell 
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timed  P-cell  ::=^x  \  s(timed  F»cell) 
total  0-value  ::=  0  |  s(total  O- value)  |  P-cell 
partial  0-value  0  |  s(O-cell)  |  total  O- value 
total  P- value  0  |  s(total  P- value) 

partial  P- value  ::=  0  |  s(P-cell)  |  0-cell 

Cells  and  values  in  these  productions  are  mixed  following  very  general  principles 
of  dialogical  interaction  that  will  be  illustrated  below  in  the  context  of  examples. 
Later,  in  discussing  how  a  play  of  the  game  for  multiplication  evolves  according 
to  such  principles,  we  shall  see  an  example  of  the  possibility  of  dynamically 
transforming  a  cell  into  a  value  throughout  a  subgame  in  which  an  auxiliary 
question  is  answered. 

The  starting  position  of  a  game  has  always  the  form 

OF{Pxu...,^Xn,‘‘y) 

with  the  Opponent  to  move.  The  labeling  of  symbols  makes  the  input  arguments 
available  to  the  Proponent  who  may  ask  the  Opponent  to  provide  (piecewise) 
their  values,  while  the  Opponent  asks  for  the  (first  piece  of  the)  output  value  of 
the  entire  expression:  the  piecewise  generation  of  values  is  intended  to  simulate 
the  process  of  generation  of  numbers.  Initially,  however,  the  Proponent  may 
require  that  some  of  the  cells  he  owns  be  timed;  the  values  with  which  timed 
cells  are  filled  are  total  P- values  s”(0)  written  completely  by  the  Opponent 
before  the  start  of  the  play.  Then  the  game  proceeds  by  alternating  0-moves 
and  P-moves,  that  may  be  either  questions  or  answers. 

Let  us  consider  for  example  the  game  associated  to  addition:  the  initial  position 
is 

°+{^x/y), 

where  the  Proponent  has  designed  the  cell  as  timed.  Then  the  Opponent 
may  make  the  move: 

o+(^x,s"(0))? 

asking  for  the  value  of  the  cell  labeled  with  his  color.  The  Proponent  has  now 
either  the  possibility  of  asking  for  the  value  of  the  P-cell  occurring  inside 
the  0-cell  which  he  is  trying  to  fill,  or  to  use  the  primitive  recursive  definition 
of  addition  applied  to  the  value  of  the  timed  P-cell  which  yields  the  partial 
0-value  s(^H-(^a;, s”"^(0)),  a  partial  answer  to  Opponent’s  original  question. 
Proponent  is  forced  to  choose  the  second  alternative,  as  the  first  leads  him  to  a 
situation  in  which  he  has  no  available  move  for  answering  Opponent’s  question. 
The  game  proceeds  with  Opponent  to  move  from  the  position 

s(O+rx,s"-i(0))). 

After  a  number  of  moves  that  is  bounded  by  the  value  with  which  Opponent  has 
filled  the  cell  the  position  in  which  Opponent  is  to  move  shall  have  the  form 

s"(O+(^x,0)). 
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The  Proponent  may  now  answer  s”(^a;),  by  considering  the  cell  s^{^x)  to  be  an 
0- value.  The  rationale  behind  this  kind  of  move  is  provided  by  regarding  a  P-cell 
as  a  promise  made  by  Opponent  to  fill  it  with  a  P-value,  Using  a  rough  analogy 
from  economy,  an  ^-cell  behaves  very  much  like  a  promissory  note  or  a  bill  of 
exchange  issued  by  ts  partner  and  owned  by  £.  Clearly,  then,  it  can  be  used  by  i 
for  discharging  a  debt  to  his  partner.  The  play  terminates,  because  Opponent  is 
now  unable  to  move  further,  hence  this  play  is  a  win  for  the  Proponent  according 
to  the  usual  termination  convention  for  games  of  this  kind.  Actually,  this  play 
describes  a  winning  strategy  for  the  Proponent  in  an  arbitrary  game  associated 
with  addition:  clearly  the  strategy  is  suggested  by  the  proof  of  Proposition  3, 
including  the  choice  of  as  a  timed  cell,  for  this  is  the  argument  up  to  which 
any  derivation  eliminating  the  addition  symbol  from  the  (term  from  which  is 
built  the)  initial  position  of  the  game  has  to  count. 

In  the  case  of  multiplication,  the  initial  position  is 

and  the  first  move  of  the  game  can  only  be  the  Opponent’s  question: 

^*(s”^(0),s«(0))?. 

The  Proponent,  being  unable  to  emit  any  value  according  to  the  primitive  recur¬ 
sive  definition  of  multiplication,  may  delay  his  answer  by  starting  an  auxiliary 
game  with  initial  position 

^+(^*(s”^(0),s”-nO)),s”‘(0)) 

and  Opponent  to  move.  Observe  that  this  move  respects  the  constraints  on  ad¬ 
missible  values  for  cells:  being  a  partial  P-value,  the  0-cell  ^♦(s’”(0),s”“^(0)) 
can  fill  the  P-cell  at  the  first  argument  place,  which  is  not  timed.  By  this  kind  of 
move,  the  0-cell  becomes  locked,  meaning  that  throughout  the  auxiliary  game 
it  is  regarded  as  a  value  by  the  Opponent  who  cannot  ask  any  question  concern¬ 
ing  this  cell.  This  amounts,  in  terms  of  (real)  dialogues,  to  Opponent  making  a 
concession  to  Proponent  which  has  however  a  limited  extent:  as  soon  as  the  Pro¬ 
ponent,  in  the  develpment  of  this  subgame,  reaches  an  O- value,  this  is  regarded 
as  the  answer  to  the  question  by  Opponent  that  started  the  current  subgame,  the 
subgame  terminates  and  the  concession  loses  its  validity  by  making  the  locked  cell 
returning  to  behave  as  an  0-cell.  In  the  present  example,  after  one  move  the  Pro¬ 
ponent  produces  the  partial  0- value  s(^-|-(^*(s”^(0),s”“^(0)),s’”“^(0))).  After 
771  such  subgames  the  position  becomes  s^(^+(*^*(s"^(0),  (0)),  0))  with  Pro¬ 

ponent  returning  the  0- value  ^*(s’^(0),s”“^(0))  which  is  then  unlocked.  The 
play  continues  from  the  position 

s”^(^*(s”^(0),s”-H0))) 

and  eventually  (that  is,  within  a  deadline  established  by  Opponent  as  the  total 
F- value  n)  the  position  becomes  s”*”(0)  with  Opponent  to  move,  a  winning 
position  for  Proponent. 
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It  is  important  that  moves  starting  an  auxiliary  game  be  allowed  only  when  an 
0-cell  is  substituted  for  a  P~cell  which  is  not  timed.  Otherwise  the  Opponent 
may  refuse  to  accept  the  move  on  the  basis  of  the  fact  that  this  is  equivalent 
to  accepting  that  the  Proponent  may  delay  his  answer  beyond  any  of  the  values 
that  are  accepted  by  the  Opponent,  and  this  clearly  causes  the  rules  to  be  biased 
towards  the  Proponent.  This  is  instead  exactly  what  happens  in  the  case  of 
exponentiation,  in  the  position 

^Px,Py'),Px). 

While  the  above  argument  outlines  a  winning  strategy  for  Proponent  in  the 
game  for  multiplication,  essentially  as  a  consequence  of  Proposition  4,  the  initial 
question  of  Opponent  cannot  be  answered  by  Proponent  in  the  dialogue  for 
exponentiation. 
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In  Plato’s  Meno  [M]  Socrates  says  at  one  point,  “The  man  who  does  not 
know  has  within  himself  true  opinions  about  the  things  that  he  does  not  know” . 
This  is  said  after  a  long  conversation  with  one  of  Meno’s  attendants,  an  unedu¬ 
cated  young  boy,  from  whom  Socrates  elicits  a  particular  version  of  Pythagoras’ 
theorem  solely  by  asking  questions. 

The  question  can  then  be  raised,  “Suppose  you  become  convinced  of  some 
<j)  solely  through  being  asked  questions  or  through  arguments  based  on  facts 
that  you  already  knew.  Does  it  then  follow  that  you  already  knew  (p  before  you 
started?”  This  is  an  old  puzzle  about  the  role  of  argument.  An  argument  is 
sound  if  the  conclusion  is  already  contained  in  the  premises.  But  if  so,  then  an 
argument  can  reveal  nothing  new  and  therefore  someone  who  knows  the  premises 
already  knows  the  conclusion. 

The  problem  of  logical  omniscience  revealed  by  this  discussion  reappears  in 
the  popular  Kripke  semantics  for  logics  of  knowledge.  Under  such  semantics,  a 
knower  i  has  an  accessibility  relation  which  is  usually  an  equivalence  relation, 
but  need  not  be  assumed  to  be  such  for  this  discussion.  Intuitively,  5  t  means 
that  the  worlds  s  and  t  result  in  the  same  state  of  information  for  i.  Then  the 
formula  Ki{(l))  holds  at  a  possible  world  (or  state)  s  iff  0  holds  at  all  t  accessible 
from  s,  i.e  at  all  t  such  that  s  Wi  t.  It  follows  immediately  that  if  (f>  is  logically 
true  then  iiCi(0)  holds  at  5,  since  it  must  hold  at  all  such  t.  Also  if  Ki{(j))  and 
Ki{(l)  — ►  hold  at  3  then  so  does  Ki{ip)?  Thus  what  i  knows  at  s  includes  all 
logical  truths  and  is  closed  under  logical  consequence. 

Given  this  state  of  affairs,  i  of  course  has  no  need  of  reasoning  since  i  al¬ 
ready  knows  everything  that  i  might  derive  through  it.  But  since  real  people  (or 
processors)  do  not  have  these  advantages,  a  more  realistic  theory  of  knowledge 
must  allow  for  a  knower  not  to  know  some  logical  truths  or  not  to  know  some 
consequences  of  things  that  she  knows.  Recent  literature  has  had  a  fair  number 


^  Research  supported  by  NSF  grant  OCR  92-08437 

^  For  both  (f>  and  ^  must  hold  at  aU  t  accessible  from  s  and  hence  'tp  must  hold 
at  all  such  t.  This  yields  the  fact  that  Ki{ip)  holds  at  s. 
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of  papers  addressed  to  this  issue  but  the  problem  is  still  largely  open.^  Our  pur¬ 
pose  here  is  to  survey  some  of  the  previous  work  and  olfer  one  or  two  suggestions. 


1.  Sentences  and  Propositions 


One  of  the  issues  that  turns  out  to  be  important  is  whether  what  is  known 
is  a  sentence,  i.e.  a  syntactic  object  in  some  language,  or  a  proposition,  which 
is  the  sense  of  such  a  sentence.  The  reason  why  this  matters  is  that  if  cj)  and  ip 
are  two  sentences  which  are  logically  equivalent,  then  they  will  denote  the  same 
proposition  and  hence  if  it  is  propositions  which  are  known,  then  knowledge  of 
(t>  implies  that  of  If  on  the  other  hand,  it  is  sentences  that  are  known,  then 
someone  who  assents  to  <l>  but  does  not  know  of  the  equivalence  may  well  dissent 
from  ij)  and  hence  we  might  want  to  say  that  he  knows  0  but  not  ip.  Thus  for 
example,  someone  who  asserts  that  his  son  is  2  years  old  might  not  want  to  say 
that  his  son’s  age  is  the  only  n  >  1  for  which  there  exist  integers  x,y,z>0  with 

4-  2/”  = 

Stalnaker  [St91]  also  discusses  the  sentence-proposition  issue,  but  decides  in 
favour  of  propositions  so  that  knowledge  of  sentences  is  not  really  discussed 
by  him.  He  points  out  that  with  the  choice  of  propositions  as  the  objects  of 
knowledge,  if  one  knows  (p  and  <j>  implies  then  one  might  not  know  -0,  but  one 
must  know  (j>  A which  is  equivalent  to  <p. 

The  issue  of  the  knowledge  of  sentences  rather  than  propositions  can  also 
arise  when  there  are  complexity  considerations.  If  we  allow  someone  polytime 
algorithms  to  decide  about  the  truth  of  something,  then  a  long  'ip  might  allow 
adequate  time  whereas  a  short  and  equivalent  <p  might  not.'^ 

This  issue  is  addressed  in  [Pa87],  one  of  the  earlier  papers  devoted  to  logical 
omniscience.  In  the  following  definition,  I  stands  for  language  and  b  for  behaviour. 

Definition  [Pa87]:  i  has  external  knowledge  oi  (p  if  (p  is  true  in  all  situations 
which  are  compatible  with  i’s  evidence. 

i  has  /-knowledge  of  ^  if  i  has  just  said  “yes”  to  the  question  whether  (p  is  true 
and  in  all  possible  situations,  i  says  “yes”  to  <p  only  if  i  has  external  knowledge 
of  <p  and  says  “no”  to  (p  only  when  i  has  external  knowledge  that  -xp  holds. 

i  has  6-knowledge  of  (p  if  there  are  three  mutually  incompatible  actions  a, 
and  7  such  that  i  does  a  only  if  (p  is  true,  does  only  if  (p  is  false,  and  moreover, 
i  has  just  done  a.® 

Here  /-knowledge  does  allow  us  to  distinguish  between  knowing  <p  and  know¬ 
ing  ip  even  when  the  two  sentences  are  equivalent.  But  for  6-knowledge,  if  i 

^  One  approach  that  has  been  tried  is  to  use  ‘impossible’  possible  worlds.  If  i  knows  (p 
and  also  <p  tp^  but  does  not  know  “tp,  then  it  is  because  there  is  an  impossible,  or 
illogical  world  from  Vs  point  of  view  where  cp  and  <p^  tp  hold  but  'tp  does  not.  This 
approach  is  discussed  in  [Li94]  but  we  shall  not  discuss  it  here  any  further. 

^  A  polynomial  applied  to  a  larger  argument  would  allow  more  time  than  the  same 
polynomial  applied  to  a  smaller  argument. 

®  7  corresponds  to  “I  don’t  know”  so  no  conditions  are  imposed  on  it. 
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6-knows  <j)  then  he  &-knows  all  equivalent  A  theromostat  that  turns  on  the 
heat  when  the  termperature  drops  to  68*^F  will  also  turn  on  the  heat  when  the 
teemperature  drops  to  20® C.  This  is  also  true  of  external  knowledge,  of  which 
the  Kripke  semantics  we  described  earlier  is  one  special  case,  since  given  that  (j) 
and  V'  are  equivalent,  evidence  for  one  is  also  evidence  for  the  other. 


2.  Implicit  and  Explicit  Knowledge  and  the  Question  of  Identity 

If  individual  i  knows  and  individual  j  knows  (j)  ^  ip,  then  together  they 
know  but  neither  might  know  by  itself.  To  bring  out  the  knowledge  of  '0 
from  the  implicit  knowledge  of  it  that  the  two  possess,  some  communication 
between  z,  j  must  take  place.  What  is  interesting  is  that  the  question  of  logical 
omniscience  even  for  a  single  individual  is  connected  to  this  distinction. 

Thus  if  a  string  x  is  written  on  the  input  tape  of  a  Turing  machine  and  the 
machine  needs  to  know  whether  x  belongs  to  some  language  L,  the  machine 
may  need  to  carry  out  a  computation.  From  the  point  of  view  of  knowledge,  this 
is  puzzling  and  connected  to  the  logical  omniscience  problem.  For  the  machine 
knows  that  the  string  on  the  tape  is  x  and  of  course  it  is  a  logical  fact  that  this 
particular  string  must  be  in  L,  if  it  is.  I.e.,  if  5  is  a  variable  over  strings  and  sq 
denotes  the  partcular  string  on  the  tape,  en  sq  =  a;  is  a  fact  that  the  machine 
knows,  and  (Vs)(s  =  x  s  e  L)  is  a,  logical  truth.  So  why  does  the  machine 
have  to  work  to  find  out  what  it  already  knows? 

But  if  we  think  of  the  string  x  ajs  written  over  many  squares  of  the  tape 
and  the  head  as  the  one  who  has  to  answer  us,  then  the  knowledge  that  x  e  L 
is  implicit  knowledge  shared  among  all  these  separate  individuals  and  naturally 
communication  among  these  (i.e.  computation)  is  needed  to  make  the  knowledge 
explicit. 

This  issue  can  be  brought  out  in  a  very  clear  way  by  the  following  example. 
Suppose  that  i  and  j  are  processors  connected  by  a  one  way  channel  from  i  to 
j.  i  knows  (p  and  j  knows  If  we  regard  z,  j  as  a  single  process  i  +  j,  then 

i  +  j  does  know  ip,  but  to  get  this  knowledge,  one  must  ask  at  the  imput  port  of 
z  and  wait  for  an  answer  at  the  output  port  of  j.  Thus  while  it  is  true  that  we 
can  regard  z  +  j  as  a  single  individual  which  knows  ip,  recovering  this  knowledge 
has  certain  side  conditions.® 


3.  Computational  Complexity 

One  approach  to  the  question  of  logical  omniscience  is  to  take  the  help  of 
complexity  theory,  which  is  also  motivated  by  the  desire  to  take  resource  limita¬ 
tions  into  account.  One  such  approach  is  that  of  Moses  [Mo88],  where  algorithms 


Indeed,  when  we  ask  a  question  over  the  telephone,  we  speak  into  the  mouthpiece 
and  expect  our  answer  ‘from’  the  earpiece 
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for  deciding  whether  one  knows  (j>  are  considered.  Such  an  algorithm  is  sound  if  it 
only  says  that  one  knows  0  when  external  knowledge  of  </>  is  present.  Knowledge 
is  now  no  longer  closed  under  logical  consequence.  For  example  there  is  a  poly¬ 
time  algorithm  for  deciding  if  a  given  graph  is  a  simple  cycle.  Call  this  property 
(j).  It  is  also  always  true  that  (^  — ►  -0  where  0  is  the  property  of  having  a  Hamil¬ 
tonian  cycle.  Hence  of  course  there  is  a  polytime  algorithm  for  0  — ►  0,  namely 
the  algorithm  which  always  says  “yes”.  However,  assuming  that  P  ^  NP,  there 
is  no  poly  time  algorithm  for  0.  Hence  it  is  possible  to  poly-know  0,  <j)  'ip,  but 
not  0.  However,  one  can  poly  time  know  0A0  when  there  is  poly  time  knowledge 
of  0,  and  0  0  is  valid. 

Moses’  approach  is  used  by  [HMT88]  who  analyse  the  zero  knowledge  pro¬ 
tocols  of  Goldwasser,  Micali  and  Rackoff  and  show  that  there  is  a  precise  tech¬ 
nical  sense  in  which  zero  knowledge  protocols  that  prove  that  x  £  L  show  only 
that  and  no  more.  Any  0  that  is  learned  as  a  consequence  of  the  dialogue  is  a 
consequence  of  x  £  L.  Their  analysis  requires  us  to  take  into  account  both  com¬ 
putational  complexity  and  probabilities,  since  these  protocols  yield  not  certain 
knowledge,  but  only  knowledge  with  high  probability. 


4.  Bounded  Rationality  and  Games 


C 


D 


Figure  I  above  is  the  problem  known  as  the  prisoner’s  dilemma.  Two  prisoners 
(row  and  column)  are  captured  and  questioned  separately.  Each  ha^  the  choice 
(C)  of  co-operating  with  the  other  prisoner  by  not  betraying  him,  and  (D)  of 
defecting,  by  testifying  against  him.  The  payoffs  in  terms  of  the  number  of  years 
of  imprisonment  are  given  in  each  box  so  that  if  both  co-operate,  they  each  get 
a  sentence  of  3  years,  but  if  they  both  defect  (D),  they  each  get  a  sentence  of  4 
years.  Clearly,  they  are  better  off  co-operating,  but  it  is  easy  to  see  that  for  each 
of  them,  D  is  better  than  C  regardless  of  what  the  other  chooses.  Thus  they  are 
both  drawn  to  an  outcome  which  is  worse  for  both  of  them.  Technically,  this  fact 
amounts  to  saying  that  there  is  a  unique  Nash  equilibrium  of  (4,4)  at  the  point 
(D,D)  even  though  the  outcome  (3,3)  at  the  point  (C,C)  is  better  for  both. 

One  solution  that  has  been  suggested  is  the  repeated  prisoners’  dilemma, 
where  the  same  game  is  played  repeatedly  and  one  might  expect  that  each  pris¬ 
oner  would  be  motivated  to  co-operate  since,  if  he  betrays  at  some  round,  he 
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would  be  afraid  of  retaliation  at  the  next  round.  However,  it  turns  out  that 
the  repeated  prisoners’  dilemma  does  not  solve  our  difficulty.  For  if  there  are  n 
rounds,  then  it  is  easy  to  see  that  neither  prisoner  has  anything  to  lose  by  de¬ 
fecting  at  the  n-th  round.  Since  there  is  no  next  round,  no  retaliation  is  possible. 
Since  defecting  at  the  n-th  round  is  the  only  rational  thing  to  do,  and  they  both 
know  this,  they  might  as  well  defect  also  at  the  n  -  1-th  round,  and  backwards 
induction  yields  that  they  will  defect  in  all  rounds. 

It  turns  out,  however,  that  if  the  two  prisoners  (or  rather  the  finite  automata 
which  model  the  prisoners  in  the  paper  [PY94])  are  not  too  smart,  then  they 
will  in  fact  do  better  than  defecting  (and  thereby  losing)  in  all  rounds.  [PY94] 
show  that  if  the  number  of  states  of  the  finite  automata  is  less  than  exponential, 
then  there  is  a  mixed  Nash  equilibrium^  which  yields  outcomes  arbitrarily  close 
to  (3,3).  So  stupidity  in  fact  pays  or  can  be  made  to  pay. 

It  is  interesting  to  point  out  that  the  use  of  finite  automata  to  model  limited 
reasoning  powers  is  not  at  all  new.  Indeed  this  was  one  of  the  motivations  behind 
the  subject  [RS59].  Both  the  pumping  lemma  and  the  Myhill-Nerode  theorem 
are  knowledge  theorems  in  disguise.  A  finite  automaton  with  n  states  can  only 
know  a  finite  amount  {log2{n)  bits)  and  hence  reading  a  long  string  it  cannot 
know  how  much  of  the  string  it  has  read.  This  yields  the  pumping  lemma  for 
regular  sets.  Similarly,  given  a  finite  automaton  with  n  states,  start  state  qi  and 
transition  function  there  is  an  equivalence  relation  defined  by  x  Wrn  y 
iff  6{qi,x)  =  6{qi,y).  The  language  L  also  defines  an  equivalence  relation 
defined  hy  x  y  iff  (iz){xz  £  L  ^  yz  £  L).  The  finite  automaton  M  with 
some  accepting  set  F  of  states  can  accept  L  iff  refines  »£,.  I.e.  iff  M  can 
know  at  least  as  much  about  strings  as  membership  in  L  requires. 


5.  Knowledge  Algorithms 

Ryle  [R]  insists  that  knowledge  how,  i.e.  possesion  of  a  skill,  is  primary  and 
that  knowledge  that,  e.g.  knowledge  of  facts,  presupposes  and  depends  on  knowl¬ 
edge  how.  The  following  definition  is  motivated  by  this  observation. 


Definition:  [Pa87]  A  knowledge  algorithm  consists  of  a  database  together  with 
a  procedure  that  takes  as  input  a  question  (say  the  truth  value  of  some  formula) 
and  some  resource  bound,  and  operates  on  the  question  and  the  database  upto 
some  point  determined  by  the  value  of  the  resource  bound.  Then  either  an  answer 
is  obtained  or  the  bound  is  exceeded.  In  the  first  case,  the  answer  is  given 
and  the  database  may  be  updated,  even  if  the  answer  depended  logically  on 
evidence  already  at  hand.  In  the  second  case  the  answer  “I  don’t  know”  is  given®. 

^  A  mixed  Nash  equilibrium  is  a  probabilistic  combination  of  pure  strategies  which  for 
a  one  round  game  would  be  just  C  and  D. 

®  This  answer  really  should  be  distinguished  from  the  same  answer  given  when  one 
knows  that  one  lacks  external  knowledge. 
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The  database  may  also  be  updated  as  a  result  of  information  received  from  the 
outside. 

Such  a  definition  does  accomodate  examples  like  those  in  Meno.  Here  is  an 
ordinary  dialogue. 

q:  Do  you  know  the  factorisation  of  143? 

r:  Not  off  hand. 

q:  Is  11  a  prime? 

r:  (After  thinking  a  little)  Yes. 

q:  Is  13  a  prime? 

r:  Yes, 

q:  How  much  is  11  times  13? 

r:  Let  us  see;  11  times  10  is  110.  11  times  3  is  33.  110 
plus  33  is  143.  Oh,  I  see. 
q:  Can  you  factorise  143  now? 
r:  Of  course,  it  is  11  times  13. 

This  dialogue  above  is  a  typical  example  of  a  situation  which  is  a  problem 
from  the  point  of  view  of  logical  omniscience,  but  not  at  all  a  problem  when  we 
see  it  in  terms  of  algorithms  and  updated  databases.  Here  the  initial  database  of 
T  may  have  no  primes  or  only  small  ones.  However,  after  this  particular  dialogue, 
the  primes  11  and  13  will  be  added  and  the  question  about  143  would  of  course 
be  answered  readily. 

A  very  similar  approach  is  adopted  by  [HMV94]  who  seem  unaware  of  [Pa87]. 
They  define  an  agent’s  local  state  to  be  a  pair  <  A,  /  >  where  A  is  an  algorithm 
and  I  is  the  rest  of  his  local  state.  In  local  state  <  A,  /  >  the  agent  computes 
whether  he  knows  0  by  applying  the  local  algorithm  to  input  (0,0-  Thus  in 
both  [Pa87]  and  in  [HMV94]  the  emphasis  has  shifted  from  knowledge  as  fact  to 
knowledge  as  procedure. 

The  puzzle  of  the  muddy  children  has  been  much  discussed  in  the  literature. 
Some  children  having  played  in  mud,  have  got  their  foreheads  dirty.  In  fact,  k  of 
them  have  dirty  foreheads.  Every  child  can  see  everyone  else’s  forehead,  but  not 
his/her  own.  The  mother  arrives  on  the  scene  and  says  “one  of  you  has  a  dirty 
forehead.”  She  then  asks  all  the  children  one  by  one,  if  it  knows  its  forehead  is 
dirty.  Strange  to  say,  they  all  say,  “I  don’t  know.” 

In  the  conventional  version  of  this  puzzle,  (see  [HM84])  all  the  children  are 
supposed  to  be  perfect  (55)  reasoners,  and  it  can  be  shown  that  the  kih.  dirty 
child  does  know  that  it  has  a  dirty  forehead,  but  with  real  children,  this  is  only 
likely  if  k  is  one.  Indeed,  everyone  knows  that  real  children  will  behave  differently 
from  idealised  children.  The  problem  is  to  find  a  theory  of  real  children. 

To  understand  the  puzzle  properly,  we  should  examine  the  conventional  ar¬ 
gument  that  the  last  dirty  child  uses  to  realise  that  its  forehead  is  dirty.  Let 
k  be  the  number  of  dirty  children.  If  fc  =  1,  then  the  dirty  child  sees  no  one 
else  who  is  dirty,  and  realises  that  it  is  itself  the  dirty  child.  Suppose  now  that 
we  have  justified  the  case  k  =  m  and  are  considering  the  case  A;  =  m  -1- 1.  The 
(m  +  l)th  dirty  child  sees  m  dirty  faces  and  knows  that  =  m  if  it  is  itself  clean 
and  /:  =  m  +  1,  if  it  is  dirty. 
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It  then  reasons:  if  A:  =  m,  then  the  child  before  me  would  be  the  last  dirty 
child  and  would  have  realised,  using  the  argument  for  k  =  m,  that  its  forehead 
was  dirty.  However,  it  said,  “I  don’t  know.”  It  follows  that  A;  =  m  +  1  and  mj/ 
forehead  must  be  dirty. 

What  happens  to  this  argument  in  a  realistic  setting? 

Suppose  that  I  am  one  of  the  children;  all  the  children  that  I  can  see,  who 
have  muddy  foreheads,  have  been  asked  if  their  foreheads  are  muddy  and  have 
already  said  “I  don’t  know.”  I  know  that  if  my  forehead  was  clean,  then  the  dirty 
child  before  me  should  have  said,  “my  forehead  is  dirty.”  Since  that  child  said 
“I  don’t  know,”  then  if  that  child  was  capable  of  making  that  inference  and  did 
not,  then  it  must  be  that  my  forehead  is  dirty. 

However,  what  if  my  forehead  is  in  fact  clean,  and  the  failure  of  the  child 
just  before  me  to  realise  that  his  forehead  is  dirty,  is  merely  due  to  a  weak 
reasoning  capacity?  In  that  case  I  should  not  assert  that  my  forehead  is  dirty. 
Thus  my  being  able  to  make  the  right  inference  about  my  forehead  depends 
on  my  trusting  the  reasoning  ability  of  the  child  before  me,  and  his  depends 
in  turn  on  his  trusting  the  children  before  him.  Also,  perhaps  he  said  “I  don’t 
know  if  my  forehead  is  dirty,”  not  because  he  lacked  the  capacity  to  make  the 
right  inference,  but  doubted  the  capacity  of  the  child  before  him.  So  the  logical 
approach  not  only  requires  each  child  to  be  a  perfect  reasoner,  but  requires  each 
child  to  assume  that  others  are  too. 

Suppose,  however,  that  I  instruct  the  children:  “I  am  going  to  put  mud  on 
some  of  your  foreheads,  and  plain  water  on  others.  I  will  put  mud  on  at  least 
one  forehead.  Then  I  will  point  at  each  of  you,  one  by  one.  If  you  do  not  see  any 
child  with  mud  on  his  forehead  that  I  have  not  pointed  to,  and  no  one  has  yet 
raised  his  hand,  then  you  must  raise  your  hand.” 

This  procedure  can  be  learned  by  the  children  much  more  easily  than  learning 
any  logic  of  knowledge.  Moreover,  if  the  children  follow  the  instructions  properly, 
then  it  will  always  be  the  case  that  the  child  who  raises  its  hand  will  be  the  last 
muddy  child.  The  earlier  proof  that  the  last  dirty  child  knows  that  its  forehead 
is  dirty  can  be  readily  converted  into  a  proof  that  the  child  that  raises  its  hand 
will  always  be  the  last  dirty  child. 

[Pa91]  discusses  other  examples  of  situations  where  reasoning  is  replaced  by 
some  sort  of  ‘unreflective’  strategy  and  it  is  shown  that  the  strategy  that  the 
logical  argument  gives  is  in  fact  the  one  that  converges  the  fastest.  However, 
even  the  fast  strategy  does  not  require  reflection,  only  ‘right  action’.  Moreover  it 
is  shown  that  there  are  also  cases  where  a  strategy  that  yields  a  high  probability 
of  being  right  converges  much  faster  than  one  that  delivers  absolute  knowledge. 

In  conclusion,  knowledge  is  best  seen  as  only  an  intermediary  to  action,  and 
probably  not  all  successful  actions  can  be  analyzed  in  terms  of  knowledge.^  That 
said,  of  course  the  notion  of  knowledge  is  very  useful  and  can  often  help  us  to 
organize  the  way  we  look  at  some  situation. 


®  For  exmaple,  it  is  not  helpful  to  say  that  someone  who  knows  how  to  ride  a  bicycle 
must  know  some  algorithm  for  doing  this  and  is  performing  such  an  algorithm. 


29 


References 

[CM86]  M.  Chandy  and  J.  Misra,  “How  processes  learn”,  Distributed  Computing 
1  (1986)  pp.  40-52. 

[Hi62]  J.  Hintikka,  Knowledge  and  Belief,  Cornell  University  Press,  1962. 
[HM84]  J.  Halpern  and  Y.  Moses,  “Knowledge  and  Common  Knowledge  in  a 
distributed  Environment”,  ACM-PODC 1984,  PP-  50-61. 

[Mo88]  Y.  Moses,  “Resource-bounded  knowledge”  in  Theoretical  Aspects  of  Rea¬ 
soning  about  Knowledge,  ed.  M.  Vardi,  Morgan  Kaufmann  1988,  pp.  261-276. 

[HMV88]  J.  Halpern,  Y.  Moses,  and  M.  Tuttle,  “A  Knowledge-based  analysis 
of  zero  knowledge”,  in  Proc.  20th  Annual  ACM  Symp.  on  Theory  of  Computing 
pp.  132-147. 

[HMV94]  J.  Halpern,  Y.  Moses,  and  M.  Vardi,  “Algorithmic  knowledge” ,  in  The¬ 
oretical  Aspects  of  Reasoning  about  Knowledge,  ed.  R.  Fagin,  Morgan  Kaufmann 
1994,  pp.  255-266. 

[Li94]  B.  Lipman,  “An  Axiomatic  approach  to  the  logical  omniscience  prob¬ 
lem”,  in  Theoretical  Aspects  of  Reasoning  about  Knowledge,  ed.  R.  Fagin,  Morgan 
Kaufmann  1994,  pp.  182-196. 

[M]  G.  Grube  (translator).  Five  Dialogues  of  Plato,  Hackett  publishing  company, 
1981. 

[Pa87]  R.  Parikh,  “Knowledge  and  the  problem  of  logical  omniscience”  ISMIS-87 
(International  Symp.  on  Methodology  for  Intelligent  Systems),  ed.  Z.  Ras  and 
M.  Zemankova,  North  Holland  (1987)  pp.  432-439. 

[Pa91]  R.  Parikh,  “Finite  and  Infinite  Dialogues”,  in  the  Proceedings  of  a  Work¬ 
shop  on  Logic  from  Computer  Science,  ed.  Moschovakis,  MSRI  publications. 
Springer  1991  pp.  481-498. 

[PR85]  R.  Parikh  and  R.  Ramanujam  “Distributed  Processing  and  the  Logic  of 
Knowledge” ,  in  Logics  of  Programs,  Proceedings  of  a  Conference  at  Brooklyn 
College,  June  1985,  Springer  Lecture  Notes  in  Computer  Science  #193.  pp.  256- 
268. 

[PY94]  C.  Papadimitriou  and  M.  Yannakakis,  “On  Complexity  as  bounded  ra¬ 
tionality”,  in  Proc.  26th  Annual  ACM  Symp.  on  Theory  of  Computing  (1994) 
pp.  726-732. 

[RS59]  M.  Rabin  and  D.  Scott,  “Finite  automata  and  their  decision  problems”, 
IBM  Jour.  Res.  Dev.  3  (1959)  114-125. 

[Ry]  G.  Ryle,  The  Concept  of  Mind,  Barnes  and  Noble  1949. 

[St91]  R.  Stalnaker,  “The  Problem  of  logical  omniscience”  Synthese  89  (1991) 
pp.  425-440. 


On  Feasible  Numbers* 


Vladimir  Yu.  Sazonov 

Program  Systems  Institute  of  Russian  Academy  of  Sciences, 
Pereslavl-Zalessky,  152140,  Russia, 
e-mail:  sazonov@logic.botik.yaroslavl.su 


Abstract.  A  formal  approach  to  feasible  numbers,  as  well  as  to  middle 
and  small  numbers,  is  introduced,  based  on  ideas  of  Parikh  (1971)  and 
improving  his  formalization.  The  “vague”  set  F  of  feasible  numbers  in¬ 
tuitively  satisfies  the  axioms  OeF,  F-f-lCF  and  2^°®°  0  F,  where  the 
latter  is  stronger  than  a  condition  considered  by  Parikh,  and  seems  to  be 
treated  rigorously  here  for  the  first  time.  Our  technical  considerations, 
though  quite  simple,  have  some  unusual  consequences.  A  discussion  of 
methodological  questions  and  of  relevance  to  the  foundations  of  mathe¬ 
matics  and  of  computer  science  is  an  essential  part  of  the  paper. 


1  Introduction 

How  to  formalize  the  intuitive  notion  of  feasible  numbers^.  To  see  what  feasi¬ 
ble  numbers  are,  let  us  start  by  counting;  0,1, 2, 3,  and  so  on.  At  this  point,  A.S. 
Yesenin- Volpin  (in  his  “Analysis  of  potential  feasibility”,  1959)  asks:  “HTiot  does 
this  ^and  so  on’  jnean?”  “Up  to  what  extent  ‘and  so  on’?”  And  he  answers:  “Up 
to  exhaustion!”  Note  that  by  cosmological  constraints  exhaustion  must  occur 
somewhat  before,  say,  which  is  larger  than  the  number  of  electrons  in 

the  universe!  In  a  stricter  sense,  2^^®  might  also  be  viewed  as  non-feasible,  but 
2^®  =  1024  is  surely  feasible.  The  problem  is  that  we  cannot  imagine  any  uni¬ 
versally  accepted  border  point  between  feasible  and  non-feasible  numbers,  seem¬ 
ingly  precluding  a  systematic  mathematical  study  of  feasibility.  Our  aim  here  is 
to  show  that,  quite  to  the  contrary,  feasibility  is  a  notion  that  can  be  captured 
and  analyzed  by  precise  mathematical  means. 

Nevertheless,  according  to  quite  a  different  approach  a  formal  border 
point  between  “feasible”  and  “non-feasible”  may  be  postulated  to  exist. 

We  just  reject  the  abstraction  of  potential  feasibility  in  another  way.  We 
could  postulate  the  existence  of  some  resource  hounds  which  always  ap¬ 
pear  in  practice  and  should  not  be  neglected,  as  usually,  but  explicitly 
taken  into  our  consideration,  say,  as  parameters.  This  leads  us  to  the 
idea  of  a  finite  row  of  natural  numbers  with  the  largest  number  (sym¬ 
bolizing  the  incidental  resource  bound),  which  may  be  denoted  like  zero 
as  □ .  It  proves  that  recursion  theory  relativized  to  such  a  finite  row  of 

*  Supported  by  Russian  Basic  Research  Foundation  (project  93-011-16016).  This  paper 
is  an  extended  version  of  the  abstract  (Sazonov  1992). 
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natural  numbers  is  essentially  the  theory  of  polynomial-time  computabil¬ 
ity  (Sazonov  1980),  (Gurevich  1983),  (Immerman  1982),  (Vardi  1982). 

Then  we  may  consider  the  corresponding  version  of  Peano  Arithmetic  in 
{0, 1, . . .  with  two  constants  0  and  □  and  the  ordinary  induc¬ 

tion  schema,  etc.  (Sazonov  1980a,  1989).  Moreover,  it  makes  sense  to  fix 
the  value  of  □  to  be  equal,  say,  to  8  (the  case  of  the  chess-board  8x8). 
However,  in  this  paper  we  will  treat  the  “negation”  of  the  abstraction  of 
potential  feasibility  somewhat  differently,  without  postulating  any  max¬ 
imal  (feasible)  natural  number. 

So,  if  we  denote  the  “vague”  or  “fuzzy”  set  of  feasible  numbers  as  F  then  we 
should  postulate  that  Oe  F,F-\-lCF  and  2^^^^  ^  F  according  to  our  intuition. 
However,  it  seems  that  Traditional  Mathematics  (both  classical  and  intuition- 
istic)  does  not  allow  considering  such  postulates  as  consistent  ones.  Also,  the 
approach  of  A. S. Yesenin- Volpin  (which  is  sometimes  called  “ultraintuitionism” 
or  “ultrafinitism”,  “actualism”  (Troelstra  1990))  being  very  suggestive,  appears 
too  informal. 

Troelstra  and  van  Dalen  (1988)  also  wrote  on  feasibility  notion  the  following. 
“Natural  numbers  are  usually  regarded  as  unproblematic  from  a  constructive 
point  of  view;  they  correspond  to  very  simple  mental  constructions:  start  think¬ 
ing  of  an  abstract  unit,  think  of  another  unit  distinct  from  the  first  one  and 
consider  the  combination  (“think  them  together”).  The  indefinite  repetition  of 
this  process  generates  the  collection  N  of  natural  numbers.  It  should  be  pointed 
out  that  already  here  an  element  of  idealization  enters.  We  regard  5,  1000  and 
10^®^°  as  objects  of  “the  same  sort”  though  our  mental  picture  in  each  of  these 
cases  is  different:  we  can  grasp  “five”  immediately  as  a  collection  of  units,  while 
on  the  other  hand  10^°^°  can  only  be  handled  via  the  notion  of  exponentiation; 
1000  represent  an  intermediate  case.  Visualizing  10^®  °  as  a  sequence  of  units  is 
out  of  the  question.  Exponentiation  as  an  always  performable  operation  on  the 
natural  numbers  involves  a  more  abstract  idea  than  is  given  with  the  generation 
of  iV.  <  . . .  >  There  are  considerable  obstacles  to  overcome  for  a  coherent  and 
systematic  development  of  ultra-finitism,  and  in  our  opinion  no  satisfactory  de¬ 
velopment  exists  at  present.”  (p.5-6.)  “. . .  certainly  we  have  much  less  difficulty 
managing  the  idealized  concept  of  the  natural  numbers,  even  though  it  is  highly 
sophisticated  one.”  (p.832.)  “On  the  other  hand,  intuitionistically  we  do  accept 
that  “in  principle”  we  can  view  10^°^°  as  a  sequence  of  units  (i.e.  we  reject  the 
ultrafinitist  objection),  and  the  authors  are  not  sure  that  this  is  really  less  seri¬ 
ous  than  the  platonist  extrapolation.  At  least  it  needs  arguments.”  (p.851.)  Also 
Borel  (1947)  mentioned  that  “the  very  large  finite  offers  the  same  difficulties  as 
the  infinite” . 

Another  informal  consideration  of  feasibility  notion  was  given  in  a  popular 
lecture  of  A.N.Kolmogorov  (1979)  whose  idea  of  middle  and  small  numbers  is 
formally  developed  in  this  paper. 

A  rigorous  mathematical  approach  to  feasibility  was  suggested  by  R.Parikh 
(1971)  and  developed  further  by  other  authors  (V.P.Orevkov  (1979),  R.O. Gandy 
(1982),  A.G.Dragalin  (1985)).  As  Professor  Gandy  noted  to  the  author,  R.Parikh 
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was  the  first  who  showed  that  feasibility  indeed  can  be  treated  as  mathematically 
coherent  notion.  However,  the  reason  for  writing  the  present  paper  is  that  his 
very  interesting  formalization  of  feasibility  notion  appears  not  to  be  completely 
adequate. 

R.Parikh  considered  the  ordinary  Peano  Arithmetic  PA  (in  the  language  of 
primitive  recursive  functions)  augmented  with  a  new  unary  predicate  F  (which 
should  not  occur  in  the  Induction  Schema  of  PA!)  and  new  axioms  like  the 
following: 

0  G  F,  IGF,  (x  G  Fky  eF=^x  +  ye  Fkx  ■  y  G  Fk^z  <  x{z  G  F)), 
and,  most  important, 

221000  ^  F. 

Here  it  is  defined  by  primitive  recursion  2o  :=  1  and  2k^i  :=  2^'=  (and  more 

2* 

generally,  2g  :=  x,  2%^^  :=  2^^  =  2f ;  2%  =  2  '  ,  k  times  “2”).  So,  231000  denotes 

a  huge  value  of  exponential  tower  of  2^®®^  number  of  stages.  The  resulting  theory 
PAir  was  proved  in  (Parikh  1971)  to  be  practically  (or  fecisibly)  consistent  in  the 
sense  that  every  formal  proof  of  a  contradiction  in  this  theory  should  contain  at 
least  2^^^°°  symbols. 

More  exactly,  it  follows  from  Parikh’s  theorems  2.2a  and  2.26  (Parikh 
1971)  that  in  any  tree-like  Hilbert-style  proof  in  PAf  of  the  contradiction 
0  =  1  the  number  of  logical  axioms  A{t)  3xA(x)  containing  F  or  their 
quantifier  complexity  or  the  number  of  new  F- axioms  involved  should 
be  >  2990. 


This  metamathematical  statement  is  proved  in  the  ordinary  mathematical  man¬ 
ner  (roughly  speaking,  in  the  framework  of  Zermelo-Prenkel  set  theory  or  the 
like)  and  is  based  on  Hilbert  and  Ackermann’s  e-symbol  elimination  technique. 
In  fact,  R.Parikh  and  other  authors  (using  also  the  cut-elimination  technique) 
are  concerned  rather  with  obtaining  complexity  estimates  for  some  proof  pa¬ 
rameters.  However,  we  prefer  to  stress  on  reasonable  concrete  values  of  these 
parameters.  We  believe  that,  e.g.  299®  or  even  2^90  non-feasible  numbers  and 
1000  is  feasible  one  in  some  absolute  sense.  So,  their  feasibility/non-feasibility 
does  not  depend  on  any  computer  technology.  (Otherwise  the  parametric  ap¬ 
proach  would  be  indeed  the  most  reasonable.)  That  is  why  we  will  often  use 
“finite” /“infinite”  instead  of  “feasible” / “non-feasible” .  We  also  call  such  num¬ 
bers  as  2^900  “imaginary  finite”  or  simply  “imaginary”  or  even  as  “infinite”. 

The  number  231000  is  too  rough  upper  bound  for  feasible  numbers  and  without 
essential  changes  of  the  above  approach  we  cannot  replace  such  upper  bound  in 
the  last  axiom  of  R.Parikh  by  2^900  (where  2^900  ^  2^000  < 

231000).  In  fact,  we  can  argue  (by  using  the  material  of  the  next  section)  that 
provability  of  2iooo  G  F  is  in  some  exact  sense  inevitable  here.  So,  even  the 
intuitively  true  axiom  2iooo  ^  F  would  be  contradictory  in  PAf-  This  means 
that  Parikh’s  upper  bound  231000  for  feasible  numbers  was  sufficiently  exact  for 
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the  concrete  formalism  he  used.  Simultaneously,  this  witnesses  that  his  theory 
(together  with  its  underlying  logic)  is  not  completely  adequate  as  a  theory  of 
feasible  numbers.  It  is  rather  a  first  satisfactory  approximation. 

This  paper  is  devoted  to  perform  some  further  step  to  overcome  this  dif¬ 
ficulty.  It  consists  in  finding  suitable  restriction  on  the  underlying  predicate 
calculus  considered  as  Logic  of  Mathematics  and  in  arguing  that  this  restric¬ 
tion  (probably^)  does  not  crucially  destroy  our  ability  to  develop  mathematical 
knowledge.  Moreover,  this  allows  to  consider  even  the  feasible  number  1000  as  in¬ 
finite  in  a  suitable  sense  (cf.  Vopenka’s  notion  of  “witnessed  universe”  (Vopenka 
1979)).  This  restriction  on  logic  (in  its  strongest  form)  proves  to  be  quite  sim¬ 
ple.  Its  main  clause  has  been  well  known  for  a  long  time  but  was  not  considered 
immediately  in  connection  with  feasible  numbers.  It  consists  just  in  rejecting  the 
cut  rule  or,  equivalently,  in  allowing  only  normal  natural  deductions  in  develop¬ 
ing  Mathematics.  The  basic  aim  of  this  paper  is  to  demonstrate  the  adequacy 
of  such  a  restriction.  We  also  suggest  some  other  more  liberal  and  still  adequate 
restriction. 

Note,  that  there  is  a  more  rough  approach  to  feasibility  which  also  was  ini¬ 
tiated  by  R.  Parikh  (1971).  Here  only  3n(2”  =  oo)  may  be  postulated,  rather 
than  2^^^®  =  oo.  There  were  many  corresponding  works  on  Bounded  Arithmetic 
where  exponentiation  is  not  a  provably  total  function.  In  addition  to  abovemen- 
tioned  (Sazonov  1980a,  1989)  we  refer  to  (Buss  1986),  (Nelson  1986)  and  to  more 
recent  books  (Hajek  and  Pudlak  1993)  and  (Krajicek  1995)  for  further  details 
and  the  literature. 

The  complexity  theorists  know  very  well  that  there  is  an  essential  difference 
between,  say,  binary  and  unary  notation  systems  for  natural  numbers.  So,  the 
(imaginary)  number  2^®°^  in  binary  notation  has  a  quite  feasible  form  100 ...  0 
(only  thousand  zeros),  but  its  unary  representation  111...  11  is  non-feasible, 
which  corresponds  to  our  intuition  about  this  number.  That  is  why  we  prefer 
(not  for  practical  aims)  unary  notation  system  which  also  properly  reflects  the 
counting  process.  It  is  very  good  that  we  also  have  binary,  decimal  and  other 
number  systems  which  allows  to  considerably  abbreviate  “unary”  numbers.  But 
this  does  not  mean  that  e.g.  each  (feasible  in  length)  binary  string  like  100 ...  0 
denotes  some  (feasible)  number.  Nonetheless,  the  tradition  is  so  strong  that 
even  in  Bounded  Arithmetic  the  abbreviations  of  natural  numbers  are  identified 
with  the  numbers  themselves.  This  theory  is  Arithmetic  only  by  the  form  of 
its  axioms.  Actually  it  proves  to  be  a  theory  of  binary  strings.  On  the  other 
hand,  Peano  and  Primitive  Recursive  Arithmetic  completely  neglect  any  such 
distinctions  because  they  are  too  strong  and  rough  for  this. 

2  Why  Consider  Feasible  Numbers? 

Let  us  first  ask  the  counter-question:  Why  consider  non-feasihle  numbers^  It 
seems  that  there  is  no  need  in  Mathematics  and  in  Applied  Mathematics  to  spe- 

^  Note,  that  we  consider  this  paper  as  a  reassuring  experiment,  a  reasonable  step,  but 
not  as  a  final  truth. 
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dally  introduce  them.  Rather,  such  unrealistic  things  as  non-feasible  numbers 
or  non-measurable  sets,  or  the  possibility  “to  make  two  apples  from  one  which 
have  the  same  form”  (due  to  the  Choice  Axiom  in  Set  Theory)  etc.  are  unde¬ 
sirable  side  effects  of  various  formal  techniques.  We  prove  by  the  Mathematical 
Induction  that  function  2®  is  total.  However,  the  computational  practice  shows 
that  it  is  actually  partial  =  oo)!  This  is  an  interesting  and  actually  well 
known  (in  Science  and  in  every-day  life)  strange  effect  when  we  immediately  see 
something  as  “black”  but  nevertheless  think  (for  a  technical  convenience,  by  a 
habit  or  for  some  other  reasons)  that  it  is  “white” . 

Another  principle  which  is  postulated  in  Mathematics  and  Logic,  despite  its 
“false”  consequences,  is  the  transitivity  of  implication.  For  example,  we  may 
argue  that  the  implication  is  not  transitive  in  some  real  situations  such  as  the 
following  one:  if  somebody  is  a  baby  today  then  he  will  be  a  baby  one  month 
later,  but  after  one  hundred  months  he  will  be  surely  not  a  baby.  Hence,  one 
hundred  applications  of  the  transitivity  of  implication  fail  in  this  case. 

It  seems  that  the  reason  for  such  approaches  to  develop  Mathematics  which 
contradict  to  our  ground  intuition  and  experience  is  in  neglecting  the  corre¬ 
sponding  “vague”  notions  “feasible” /“non-feasible”,  “big” /“small”,  etc.  as  non- 
mathematical  ones.  Also  the  resulting  ordinary  working  apparatus  proves  to  be 
still  extreemly  successful,  sufficient  and  adequate  in  many  other  important  re¬ 
spects.  However,  why  should  we  consider  these  traditional  approaches  as  the 
best  or  the  unique  possible  ones? 

Note  that  the  ordinary  Complexity  Theory  also  deals  with  feasibility  prob¬ 
lems.  Therefore,  for  completeness’  sake  this  important  comparatively-quantita- 
tive  approach  to  algorithms  theory  might  be  deliberately  concerned  also  with 
feasible  numbers  (not  just  only  with  feasible  computations  of  functions  and  pred¬ 
icates  defined  both  on  feasible  and  non-feasible  arguments).  Kolmogorov’s  Com¬ 
plexity  Theory  of  Finite  objects  seems  was  sufficiently  close  to  this  idea.  Probably 
its  highest  success  in  the  traditional  framework  was  the  reason  that  this  theory 
did  not  turn  to  feasibility  considerations  in  a  rigorous  mathematical  way. 

Moreover,  by  the  author’s  opinion  feasible  numbers  could  be  the  proper  no¬ 
tion  to  set  into  the  foundation  of  (Applied)  Mathematics.  This  is  simply  another 
way  to  introduce  complexity  theoretic  approach  in  Mathematics  by  reconsider¬ 
ing  the  initial  fundamental  notions.  It  seems  that  this  could  give  more  smooth 
connection  of  Mathematics  with  real  computers.  Also  this  is  a  different  and 
hopefully  more  natural  approach  to  so  called  “fuzzy”  Mathematics  as  well  as 
to  (Feasibly)  Constructive  Mathematics.  As  is  well  known,  the  Ordinary  Con¬ 
structivism  allows  transformation  of  existence  proofs  to  potential  constructions 
of  corresponding  “existing”  objects.  In  contrast.  Feasible  Constructivism  should 
guarantee  just  feasible  constructions  of  feasible  objects.^ 

^  There  are  also  other  approaches  to  Feasible  Constructivism  or,  more  precisely,  to 
Polynomial  Constructivism  (Cook  1975),  (Cook  and  Urquhart  1993),  (Buss  1986), 
(Safonov  1989)  which  are  concerned  with  extracting  from  constructive  existence 
proofs  the  corresponding  polynomial-time  constructions  of  possibly  non-feasible  (in 
the  proper  sense)  finite  objects. 
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Let  us  illustrate  this  on  the  following  quasi-practical  example.  Consider  a 
variation  of  the  chess  game  which  differs  from  the  ordinary  one  essentially  by 
allowing  for  whites  and  blacks  to  make  just  two  moves  of  the  pieces  at  once 
instead  of  one  move.  Let  also  the  overall  number  of  moves  is  bounded,  say,  by 
100.  Then  we  may  easily  prove  that  whites  have  a  strategy  which  allows  them 
at  least  not  to  lose  the  game.  Indeed,  otherwise  they  can  move  a  knight  forth- 
and-back  after  which  blacks  prove  to  be  at  the  symmetrical  position!  Intuitively, 
it  is  clear  that  this  proof  of  the  existence  of  a  strategy  for  whites  is  highly  non¬ 
constructive.  (For  the  ordinary  chess  game  we  have  no  proof  at  all!)  On  the 
other  hand,  from  the  point  of  view  of  the  traditional  constructivism  we  may 
potentially  find  (by  successive  trials)  the  required  strategy  for  whites  which  may 
be  considered  even  as  a  (huge)  finite  object  of  a  bounded  size.  We  believe  that 
a  Feasible  Number  Theory  is  a  reasonable  framework  which  will  give  a  precise 
sense  to  the  notion  of  Feasible  Constructivity.  Only  then  we  could  hope  to  prove 
the  plausible  hypothesis  that  there  exists  no  feasibly-constructive  proof  for  (the 
variation  of  or  for  the  original)  chess  game  that  whites,  say,  have  a  winning  or 
non-losing  strategy. 

If,  nevertheless,  whites  do  have  an  intuitively  feasibly  constructive  strategy 
in  any  reasonable  rigorous  sense  then  this  could  be  guaranteed  just  by  a  proof  in 
a  Feasibly  Constructive  Theory.  Not  only  the  ordinary  Constructivism,  but  even 
Polynomial  Constructivism  mentioned  in  the  above  footnote  can  do  nothing  in 
this  situation.  It  probably  could  work  if  we  generalize  8x8  chess-board  to  nxn 
for  sufficiently  large  n.  However,  what  about  8x8? 

3  Formal  Systems  Revised 

What  is  (a  proof  in)  a  formal  or  an  axiomatic  system?  It  is  necessary  here  to  give 
a  right  answer  to  this  question.  The  ordinary  explanation  of  this  notion  is  rather 
rough  for  our  aim  to  formalize  feasible  numbers.  We  define  a  formal  system 
e.g.  as  a  finite  set  of  rules  Ai . .  .Ak I Ak+i  where  Ai  are  some  syntactically  well 
formed  (schemes  of)  formulas.  However,  we  may  consider  finite  (instances  of  the 
schematic)  formulas  Ai  and  their  sequences  (which  are  formal  proofs  according 
to  the  rules)  in  three  ways: 

1.  as  some  real  or  feasible  strings  of  symbols  which  may  appear  on  a  sheet  of 
paper  or  in  a  computer  memory, 

2.  as  some  abstract,  imaginary  finite  strings  which  are  considered  only  as  poten- 
tially-feasible^ ,  or 

3.  inside  some  mathematical  (meta-)  theory  (such  as  PA  or  ZF  via  a  Godel 
numbering  or  the  like). 

In  the  first  two  cases  our  intended  subject  is  some  axiomatized  branch  of 
Mathematics  described  by  the  given  formal  system  and  developed,  respectively, 

^  As  usual,  this  term  does  not  mean  something  which  could  be  made  feasible  in  our 
sense.  It  rather  allows  to  think  about  too  long  non-feasible  strings  which  are  consid¬ 
ered  as  finite  only  by  some  sufficiently  formal  reasons  or  by  an  idealization. 
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in  a  strictly  formal  or  potentially  formal  way.  In  the  third  case  we  are  actually 
considering  Metamathematics  of  the  formal  system,  so  making  more  precise  the 
second  case.  (However,  it  may  be  asked,  which  way  we  prove  metameta. .  .ma¬ 
thematical  results?  Let  us  interrupt  this  infinite  regress!)  Of  course,  all  these 
aspects  are  explicitly  or  implicitly  involved  in  everyday  mathematical  activity. 
But  the  most  essential  point  of  this  paper  is  that  we  should  not  mix  them. 

Let  us  consider  for  example  the  following  first  order  theory  T  (regarded 
eventually  as  a  finite  list  of  formal  axiom  schemes  and  rules)  of  some  weak 
arithmetic  of  natural  numbers  (even  with  no  Induction  Axiom  at  all).  Non- 
logical  symbols  of  T  are  one-place  function  symbol  s  for  the  successor  operation 
s(x)  =  ar  -f  1  and  three-place  predicate  symbol  R  with  the  meaning  R(xy  y^z) 
x  -h  2^  =  ^.  There  are  only  two  special  axioms  in  T  recursively  defining  R: 

^  (R{x,0,sx)  {x  +  2^=x-\-l)  and 

1  z)kR{z,  y,  v)  =>  R(x,  sy,  v)  (x  +  2^+^  =  {x  +  2^)  -h  2y), 

Note,  that  T  does  not  prove  that  x  -f  2^^  is  a  total  function,  i.e.  formally 

T  y- '^xy3zR(xjy,z). 

Define  the  following  sequence  of  formulas.  Eo{x)  :=  x  =  x;  Ei^i{x)  :=  3y  G 
Ei.R{0,x,y)  (:=  By{Ei{y)SzR{0,x,y))  ^  “2®  is  defined  and  6  E”).  Hence  Ei{x) 
means  that  the  value  of  2f  is  defined.  Also  take  iVo(a;)  \=  x  =  x  and  Nij^i{x)  := 
Vy  G  N^z  G  NiR{y,x,z)  {^Vy  G  Ni{y-^2^  G  AT^)). 

Theorem.  T  h  Eiooq(0)  (i.e.  T  \-  ^^looo  is  a  finite  number”). 

Proof.  (Essentially  due  to  V.P.Orevkov  (1979);  cf.  also  R.Statman  (1978, 1979).) 
We  first  infer  ^^(0),  i  =  0,1,2,...,  in  T.  For  z  =  0,1  this  is  trivial.  The  case 
Ni+2{0)  is  equivalent  to  proving  the  formula  ^y  G  Ni^i(y  -f  1  G  Ni+i)  or  equiv¬ 
alently  yyl\/x  G  Ni{x  -h2y  e  Ni)  Va;  G  Ni{x  2^+^  G  Ni)].  But  the  latter 
follows  from  the  second  axiom  of  T. 

Then  we  can  prove  Ni  C  Ei  in  T  by  induction  on  i  =  0, 1, 2, . . ..  The  case 
i  =  0is  trivial.  To  prove  Ni+i  C  Ei^i  we  take  any  y  G  iVi+i,  i.e.  any  y  such  that 
(2^  is  finite  and)  Ni  is  closed  under  addition  of  2*^,  and  apply  it  to  0  G  Wi.  This 
gives  2^  =  0  +  2^  G  iVi  C  Ei  and  hence  y  G  £?t+i)  as  required. 

It  follows  step-by-step  that  all  Ei(0),  i  =  0, 1, 2, . . . ,  1000,  are  provable  in  T. 

□ 

We  claim  that  there  is  something  wrong  in  the  above  proof  (which,  however, 
seems  very  nice  in  itself).  Indeed,  what  and  how  have  been  proved  here?  First 
of  all,  Mathematics  and  Metamathematics  were  mixed  strongly.  Even  in  the  for¬ 
mulation  of  the  theorem  the  expression  Eiooo(O)  is  not  a  formula  of  our  language 
but  only  a  short  denotation  for  some  legitimate  but  rather  long  formula.  {Ei{x) 
contains  exactly  3  -h  13i  symbols). 

However,  this  is  not  the  main  difficulty  with  this  proof.  Of  course,  Eiqoq{0) 
could  be  eventually  written  explicitly  (13003  symbols  are  not  so  many).  Much 
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worse  is  the  case  with  the  (recursive)  abbreviation  iViooo  which  cannot  be  elimi¬ 
nated  in  practice  because  the  intended  formula  of  the  original  language  evidently 
should  contain  symbols.  So,  this  direct  attempt  to  make  the  proof  rigor¬ 

ous  i.e.  to  eliminate  the  Abstraction  of  Potential  Feasibility,  Metamathematics 
and  other  informal  and  illegal  means  does  not  succeed. 

The  conclusion  is  that  the  formula  (denoted  by)  Eiooo(O)  was  not  feasibly 
proved  in  T.  We  only  proved  that,  potentially  or  metamathematically,  there 
exists  an  imaginary  finite  proof  of  the  formula  jKiooo(O)  in  T.  We  cannot  be 
completely  satisfied  by  this  metaproof  of  proof  existence  because  we  strongly 
believe  that  the  genuine  mathematical  proof  should  be  sufficiently  short  to  be 
really  written  e.g.  in  a  book. 

A  reasonable  way  of  eliminating  “meta”  from  metaproofs  consists  in  ex¬ 
tending  the  underlying  predicate  logic  to  legitimate  some  formalism  for  required 
abbreviations.  This  is  a  way  to  replace  intuitive  and  metamathematical  means 
by  formal  mathematical  ones  which  we  will  adopt  here^.  Therefore  we  consider 
(some)  abbreviation  mechanisms,  in  general,  as  strong  mathematical  (rather 
than  logical)  tools. 

Mathematical  principles,  in  contrast  to  logical  ones  (as  we  understand  them 
here),  may  have  some  special  consequences  about  objects  under  consideration 
(e.g.  the  existence  of  very  large  numbers  etc.).  The  above  Theorem  and  especially 
considerations  below  show  that  some  kinds  of  abbreviations  indeed  may  have 
such  consequences®. 

Of  course,  we  cannot  give  the  most  general  exhaustive  mechanism  for  abbre¬ 
viations  to  be  used  in  Mathematics.  This  seems  quite  analogous  to  our  inability 
to  give  a  complete  formalization  of  arithmetic  or  set  theory.  But  we  may  in¬ 
troduce  some  useful  and  concrete  such  mechanisms^.  Also  we  would  not  try  to 
find  abbreviations  as  strong  as  possible.  Just  as  we  choose  some  axioms  and 
reject  others  in  developing  some  branches  of  Mathematics,  we  will  prefer  only 
those  most  adequate  abbreviation  mechanisms  which  do  not  prevent  us  from 
formalizing  the  subject  under  consideration. 

Remember  that  our  present  subject  is  feasible  numbers.  And  the  above  the¬ 
orem  shows  that  theory  T  together  with  all  abbreviation  mechanisms  used  in  it 
(both  explicitly  mentioned  above  and,  even  more  important,  implicit  ones)  is 
non-adequate  for  this  aim:  it  proves  that  2iooo  is  finite  (what  means  here  ‘feasi- 

®  Another  idea  to  introduce  Ei  and  iVi,  i  =  1 . . .  1000,  immediately  in  the  language  of 
T  and  to  consider  their  definitions  as  a  new  axioms  of  T  seems  not  very  appropriate. 
We  must  not  change  a  given  theory  and  its  language  depending  on  theorems  to  be 
proved.  Let  us  play  in  a  game  with  fixed  rules! 

®  In  fact,  the  abbreviations  Ei  and  Ni  in  the  above  theorem  prove  to  be  not  so  crucial 
in  this  respect.  We  concentrated  on  them  just  to  show  the  role  of  abbreviations  in 
the  general  notion  of  formal  proof.  Yet  more  important  for  us  is  some  other  kind  of 
abbreviations  which  were  used  in  the  above  theorem  implicitly.  They  are  discussed 
below. 

^  Actually,  in  this  paper  we  only  mention  such-and-such  abbreviation  mechanisms 
without  introducing  them  rigorously.  What  we  will  do  here,  is  putting  a  formal  veto 
on  some  such  mechanisms. 


38 


ble’).  However,  intuitively  even  2^°®®  <  2iooo  should  be  infinite!  Note,  that  this 
consideration  on  theory  T  may  be  repeated  for  Parikh’s  formalization  PA/?  of 
feasible  numbers  mentioned  in  the  Introduction  above.  I.e.  there  exists  a  feasibly 
long  proof  in  PA/?  (with  some  kind  of  abbreviations  used,  as  above)  of  2iooo  €  F, 
It  follows  that  Parikh’s  upper  bound  221000  ^  F  cannot  be  strengthened  even 
to  2iooo  ^  F  without  essential  reconsidering  the  whole  approach.  Note  that  no 
general  restriction  on  abbreviating  means  of  PA/?  was  imposed  in  (Parikh  1971) 
(except  those  rather  technical  requirements  on  proofs  which  we  mentioned  in 
the  Introduction). 

Abbreviations  may  be  applied  not  only  to  formulas,  but  also  to  proofs.  For 
example,  in  the  above  theorem  the  proof  of  C  Ei^i  was  described  using 
recursively  its  subproof  of  Ni  C  Ei,  and  without  this  the  resulting  proof  of 
A^iooo  Q  Fiqqo  should  be  rather  long  (instead  of  a  proof  occupying  only  a  quarter 
of  page). 

However,  the  most  crucial  abbreviations  widely  used  in  Mathematics  deal 
with  terms  and  objects  (in  comparison  with  formulas  and  proofs  as  above).  Let 
us  consider  e.g.  the  simplest  term  abbreviation  2-x  :=  x^-x.  Then  we  may  denote 
the  number  2^°°°  as2'2-2-...-2-l  (thousand  times  ‘2’).  This  denotation,  being 
rather  long,  is  nevertheless  quite  feasible.  However,  it  is  impossible  to  really 
denote  (even  in  a  computer  memory)  such  extraordinarily  large  number  using 
only  1  and  +,  because  this  requires  2^®°®  occurrences  of  I’s.  This  suggests  that 
abbreviation  of  terms  is  not  an  appropriate  tool  if  we  want  to  formalize  feasible 
numbers  (and  thereby  to  exclude  2^®®®  from  this  numbers). 

Note,  that  the  ordinary  formalizations  of  the  predicate  logic  contain  implic¬ 
itly  some  kind  of  abbreviations  of  terms.  In  the  case  of  the  Natural  Deduction 
Calculus  (Prawitz  1965)  we  have  the  rules  of  introduction  (I)  and  elimination 
(E)  for  each  logical  connective,  for  example  for  3: 


m 

^xA(x) 


(31) 


3xA{x) 

B 


[A{x)] 

V 

—  (3E) 


where  x  is  not  free  in  B  and  in  the  open  assumptions  in  except  [A{x)],  and 
quantification  is  understood  up  to  proper  renaming  the  quantified  variable.  So, 
the  first  rule  abbreviates  (possibly  rather  long®)  term  t  by  the  name  x.  The  second 
one  uses  the  name  x  for  some  object  satisfying  A.  If  in  a  natural  deduction  some 
3-formula  occurrence  is  both  the  conclusion  of  31-rule  and  the  main  premise  of 
3E-rule  then  x  plays  the  role  of  an  abbreviation  of  a  term  which  is  used  in  a 
deduction.  We  claim  that  these  are  such  situations  (likewise  the  abbreviation 
2'x:=x^-x)  which  give  rise  to  non-feasible  numbers  and  therefore  they  should 
be  avoided.  We  will  avoid  analogous  situations  for  other  logical  connectives  as 
well:  introduced  by  an  I- rule  logical  connective  is  not  allowed  to  be  eliminated  by 
the  corresponding  E-rule.  The  reason  is  that  such  subinference,  for  example, 

®  and  even  if  not  long,  what  about  the  iteration  of  such  abbreviations? 
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A  B 
AkB 
A 


(&I) 

(&£) 


may  lead  to  the  above  situation  with  existential  quantification  when  the  formula 
A  is  3xA{x)y  its  upper  occurrence  in  the  figure  shown  is  the  conclusion  of  some 
31-rule  and  its  lower  occurrence  is  the  main  premise  of  3E-rule. 

Slightly  generalizing,  this  means  that  our  restriction  on  proofs  will  consist 
in  allowing  only  normal  natural  deductions  (cf.  the  exact  definition  in  (Prawitz 
1965)  and  (Troelstra  and  van  Dalen  1988)).  In  particular,  this  also  means  that  we 
can  not  freely  use  the  general  modus  ponens  rule  (=>  E)  with  the  corresponding 
rule  {=>  I) 

[^] 


because  the  premise  A  =>>  B  of  the  former  could  be  introduced  in  the  deduction 
by  the  latter  so  that  the  normality  requirement  fails  (and  again,  as  above,  it  may 
be  considered  the  case  when  B  is  3xB{x)). 

In  this  connection  it  is  worth  to  remember  an  anticipating  note  in  (Troelstra 
and  van  Dalen  1988):  “The  strictly  finitist  view  also  has  its  consequences  for 
logic;  the  derivations  of  A  and  A  B  may  still  be  within  reach,  but  in  order 
to  apply  modus  ponens  one  might  have  to  exceed  the  available  natural  numbers 
necessary  for  the  length  of  the  derivation  of  B.”  (p.29.) 

Similarly,  there  is  no  guarantee  that  implication  is  transitive.  This  might 
seem  very  strange  if  we  forget  that  our  aim  is  to  formalize  such  vague  notion 
as  feasible  numbers  or,  say,  Vopenka’s  notion  of  the  horizon  (Vopenka  1979) 
or  the  notion  of  middle  or  intermediate  numbers.  The  latter  notion  could  be 
considered  as  natural  numbers  counted  before  the  horizon  is  “overcomed” .  Here 
0,1, 2,...,  10, 11,...  are  middle,  1000  and  even  100  are  definitely  not  middle 
(and  lie  “behind  the  horizon”)  and  there  is  no  maximal  middle  number.  (Just 
look  along  a  straight  railway  towards  the  horizon  and  count  the  pillars;  see  also 
the  discussion  in  §  2  on  non- transitivity  of  implication  and  §  4  below.) 

Of  course,  every  natural  deduction  can  be  (potentially!)  normalized  (Prawitz 
1965).  However,  it  is  well  known  that  this  normalization  (or  cut  elimination) 
process  has  non- elementary  lower  (and  upper)  complexity  hounds  (Orevkov  1979; 
Statman  1978,  1979).  For  example,  consider  the  above  proof  of  Biooo(O)  in  the 
Natural  Deduction  form: 

[•^1000  (0)] 

:  ^1000  (0)  jx 

-^1000(0)  iViooo(O)  Eiqoq{0) 

- WO) 
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It  is  not  normal^  because  the  conclusion  of  (=>  I)-rule  is  the  main  premise  of 
(=^  E)-rule.  This  proof  cannot  be  normalized  in  practice.  Indeed,  every  normal 
proof  of  the  formula  Eiooo(O)  (which  asserts  the  existence  of  the  number  2iooo) 
will  contain  a  term  of  the  length  >  2iooo  essentially  due  to  the  following  form  of 

Herbrand’s  metatheorem.  Every  normal  classical  deduction  of  ^-formula 
3zCz  or  ~-i->3-formula  ~>~i^zCz  (possibly  without  3)  from  V-  and  ~i3-formulas 
of  the  kind  "ixAx  and  -i3yBy  can  be  reconstructed  into  quantifier-free  normal 
deduction  of  some  finite  disjunction  ViCU  from  the  formulas  of  the  kind,  re¬ 
spectively,  At  and  -tBs,  and  conversely.  Moreover  (^and  most  crucial^,  in  both 
directions  the  new  deduction  will  contain  only  those  terms  which  were  occurring 
in  the  initial  one.  □ 

Nevertheless,  using  abbreviations  is  extremely  important  tool  of  Mathemat¬ 
ics.  Therefore,  it  would  be  not  very  reasonable  to  reject  all  of  them.  That  is  why 
we  summarize  our  special  requirements  on  mathematical  proofs  by  the  following 
three  rather  informal  clauses  (except  the  clause  2): 

1.  Arbitrary  explicitly  fixed  abbreviation  mechanisms  for  formulas 
and  proofs,  but  not  for  terms  are  allowed. 

2.  Only  normal  proofs  are  allowed. 

3.  The  number  of  symbols  in  a  proof  should  be  (intuitively)  feasible. 

Of  course,  these  requirements  could  be  formulated  more  rigorously  and  also 
in  a  more  strong  or,  on  the  contrary,  in  a  more  weak  form.  For  example,  in 
3)  ‘feasible’  could  be  strengthened  by  ‘middle’  or  something  such,  because  the 
genuine  mathematical  proofs  should  be  clear  and  hence  not  only  feasible,  but 
also  rather  short. However,  it  will  be  quite  sufficient  for  the  current  aims  and 
for  simplicity  sake  to  weaken  our  requirements  as  follows: 

ONLY  NORMAL  PROOFS  WITH  FEASIBLE  TERM  SIZE 
ARE  ALLOWED 

Here  term  size  of  a  proof  is  defined  as  the  maximum  of  the  number  of  symbols 
in  each  term  occurrence  in  the  proof.  No  restriction  is  imposed  on  the  number 
of  term  occurrences  and  on  the  length  of  proofs  and  formulas  in  this  final  re¬ 
quirement.  So,  proofs  and  formulas,  except  terms,  may  be  treated  abstractly,  as 
potentially  feasible.  This  considerably  simplifies  the  matter,  and  the  resulting  re¬ 
quirement  on  proofs  naturally  corresponds  to  the  above  three  clauses.  However, 
we  believe  that  the  ideal  approach  should  be  based  on  those  clauses,  probably 
with  the  technical  rather  strong  restriction  2)  replaced  by  some  more  liberal  and 
hopefully  more  convenient  one;  cf.  also  §  5  below. 

®  and,  in  fact,  uses  abbreviations  of  terms  via  quantifier  rules  here  not  shown 
There  are  many  examples  when  rather  complicated  mathematical  proofs  became 
very  transparent  and  rather  short  after  suitable  reconsidering  the  presentation  of 
the  whole  theory.  After  all,  the  main  idea  of  any  mathematically  interesting  proof  is 
usually  sufficiently  simple. 
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Instead  of  the  natural  deduction  we  could  formalize  mathematical  proofs  by 
Gentzen’s  sequent  calculus.  In  this  case  the  normal  proofs  may  he  equivalently 
replaced  hy  cut-free  ones.  However,  we  prefer  natural  deduction  just  because 
it  is  “natural”  and  our  aim  is,  after  all,  to  develop  Feasible  Mathematics  with 
as  minimum  as  possible  extra  technical  efforts  (which,  however,  are  inevitable) 
connected  with  choosing  any  unnatural  formalism  for  a  real  deducing  theorems. 

Let  r  hy  A  mean  that  there  exists  some  (possibly  imaginary)  normal  clas¬ 
sical  first-order  natural  deduction  (or  cut-free  sequent  deduction)  of  A  from  JT 
with  intuitively  feasible  term  size.  To  assert  F  \-^  A,  it  is  sufficient  to  be  able 
to  really  write  down  each  term  occurrence  in  the  corresponding  proof  or  even  to 
be  surely  convinced  that  such  a  proof  with  short  terms  exists.  F  A  will  mean 
that  there  exists  no  such  proof.  We  may  be  quite  sure  about  this  if,  for  example, 
we  have  some  traditional  meta-proof  (in  ZF  or  the  like)  that  there  is  no  required 
proof  of  the  term  size  less  than  2^°°°.  As  an  additional  technical  convention  we 
will  consider  the  negation  sign  as  definable  one: 

-lA  :=  (A  X), 

where  X  is  the  primitive  logical  symbol  denoting  falsity  with  the  ordinary  reduc- 

tio  ad  absurdum  rule  of  inference  (Prawitz  1965):  [-»A] - L/A.  In  particular,  we 

have  the  inference  rule:  X/everything.  We  will  see  below  that  this  convention 
about  negation  plays  an  essential  role  in  the  consistency  of  a  theory  of  feasible 
numbers  FEAS  defined  in  the  next  section. 

It  might  be  thought  that  the  above  notion  of  nf-proof  is  too  vague  due  to 
the  involved  intuitive  notion  of  feasible  terms.  However,  the  implicit  use  of  the 
abstraction  of  potential  feasibility  of  proofs  in  the  ordinary  approach  to  the 
notion  of  proof  seems  much  more  unclear.  We  believe  that  the  only  way  of  rig¬ 
orous  formalization  of  Mathematics  is  through  feasible  proofs  (in  various  formal 
systems).  On  the  contrary,  the  potentially  feasible  proofs  in  the  full  generality 
of  this  notion  (if  it  hcis  any  sense  at  all)  hardly  can  be  considered  as  a  genuine 
formal,  i.e.  mathematical  one  due  to  the  “infinite  regress”  implicit  in  this  notion. 

Note,  that  we  are  speaking  here  only  about  the  formal  and  routine  nature 
of  mathematical  proofs.  How  these  proofs  and  corresponding  formal  sys¬ 
tems  are  created  is  quite  different  question.  Of  course,  this  is  usually  ex¬ 
tremely  informal  process.  Nevertheless,  any  mathematical  result  should 
be  presented  in  a  sufficiently  formal  way.  It  is  very  important  that  the 
ordinary,  not  completely  formal  proofs  in  Mathematics  usually  can  be 
transformed  into  feasibly  formal  ones  (as  the  above  proof  of  £7iooo(0))  via 
explicating  the  necessary  abbreviating  mechanisms.  The  author’s  opinion 
is  that  it  is  this  reason  why  any  particular  not  very  formal  mathemati¬ 
cal  proof  is  actually  considered  by  mathematical  community  as  genuine 
mathematical  one. 

The  reader  might  probably  consider  as  a  rather  artificial  the  above  normality  re¬ 
quirement  of  proofs  (taken  simultaneously  with  the  requirement  on  feasibility  of 
term  size).  This  also  seems  too  strong  restriction  to  the  author  himself.  However 
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note,  that  our  aim  was  to  find  first  any  reasonable  restriction  which  allows  to 
formalize  feasibility  sufiiciently  adequately.  On  the  other  hand,  we  will  present 
in  §  5  some  more  liberal  restriction  on  the  predicate  calculus  which  represents 
more  directly  our  main  idea  that  only  terms  must  not  be  abbreviated.  There 
may  be  also  some  other  approaches  and  variations. 

4  A  Basic  Theory  of  Feasible  Numbers 

The  following  theory,  FEAS,  is  a  point  of  departure  for  formalizing  feasible  num¬ 
bers.  The  theory’s  non-logical  symbols  are  0, 1,  +,  [log2 . .  .J  and  <.  Let  FEASq 
denote  a  collection  of  closed  universal  formulas  (with  terms  of  intuitively  feasible 
length)  which  are  feasibly  true  in  an  intuitive  sense,  such  as  ^x{x  ^  x  -fl  ^  0), 
VajVj/  ^  0(a;  <  log2  2/  =>  x  +  1  <  log2(2/  +  ?/)),  log2  1  =  0  and,  for  definiteness, 
log2  0  =  0,  i.e.,  several  ordinary  axioms.  We  will  assume  that  this  collection 
contains  the  feasibly  true  universal  formulas  that  we  need.  Now  define  FEAS  to 
be  FEASo  extended  with  the  Main  Axiom 

V2/(log2  log2  2/  <  10), 

that  is,  2^  =  2^°^^  =  oo;  this  too  is  a  universal  formula,  and  is  intuitively  true 

for  feasible  numbers  (in  a  new  natural  sense  respecting  the  old  one).  Those  in 
doubt  may  wish  to  check  this  on  a  computer  for  various  feasible  y^s  represented 
in  unary  notation. 

Now  we  assert  the  following  facts  about  FEAS. 

Fact  1.  FEAS  y-^  X. 

This  assertion  holds  because  any  normal  proof  of  X  in  FEAS  involves  a  term 
which  is  too  long  to  be  physically  written  down  or  stored,  namely,  we  have 

Metatheorem  for  FEAS.  Every  normal  proof  of  X  in  FEAS  contains  a  term 
with  >  2^^^^  symbols. 

Proof.  By  Herbrand’s  Theorem,  in  the  form  presented  in  §  3,  each  universal 
axiom  of  FEAS  that  occurs  in  a  normal  proof  of  X  can  by  replaced  by  closed 
instances  thereof.  The  value  of  each  closed  term  in  the  language  of  FEAS  is 
bounded  by  its  size.  Substitution  instances  of  axioms,  including  the  Main  Axiom, 
would  be  true  in  the  standard  sense  if  all  substituted  terms  were  of  length  < 
21024  Therefore,  true  (in  the  standard  sense)  axioms  would  imply  X,  which  is 
impossible.  □ 

Another  important  corollary  of  Herbrand’s  Theorem  is: 

Fact  2.  The  theory  FEAS  w  a  conservative  extension  of  FEASq  with  respect  to 
closed  quantifier-free  and  {-t->)3-formulas.  In  fact,  if  a  {-^-^3) -sentence  has  a 
proof  in  FEAS  of  term  size  <  2^°24^  ^  FEASq  of  the  same 

term  size.  □ 


This  means  that  the  two  theories  prove  the  same  theorems  about  the  ter- 
minaton  of  computations,  that  is,  the  kind  of  existential  statements  that  is  of 
greatest  value  in  applications.  Theorems  of  other  forms,  such  as  the  Main  Axiom 
V2/(log2  log2  2/  <  10),  are  aimed  at  providing  a  reasonable  abstract  context  for 
computations  and  algorithms  (as  in  Hilbert’s  Program).  Nonethelss,  we  have 

Fact  3.  FEAS  +  Va;(/(rc)  =  x  H-  x)  hj  ±,  where  f  is  a  new  function  symbol  for 
multiplication  by  two.  D 

Thus  the  practically  consistent  theory  FEAS  becomes  inconsistent  once  a 
name  for  the  doubling  function  is  introduced.  This  example  have  been  discussed 
in  §  3  above.  Note  that  term  size  of  the  proof  in  Fact  3  is  1000  symbols.  The 
details  are  left  to  the  reader. 

Let  us  define  M{x)  :=  “x  is  a  middle  (or  intermediate)  number'^  :=  3y  ^ 
0(x  <  log2  y)  (here  i/  ^  0  is  an  inessential  technical  restriction  to  simplify  one 
formal  proof  below)  and  S{x)  :=  “x  is  a  small  number''  :=  3y{x  <  log2log2  2/)- 
Then,  we  have 

Fact  4.  FEAS  5(0),  -5(10),  3x(5(x)&-5(x  +  1)),  M(0),  -.M(1024), 
Vx(M(x)  M(x  +  1)),  Vx  <  y{S{y)  =>  5(x)),  Vx  <  y{M{y)  ^  M(x))  and 
Vx(5(x)  =>  M(x)). 

Note  that  provability  of  M(0), -M(1024)  and  Vx(M(x)  =>  M(x  +  1))  gives 
no  contradiction  here.  Indeed,  the  reader  may  see  that  the  corresponding  deduc¬ 
tion 

M(0),  M(0)=J^M(1),  Af(l),  M(1)=^M(2),  M(2),  ...,  M(1024) 

by  multiple  application  of  modus  ponens  rule  is  not  normal  one  because  M(x)  => 
M(x  +  1)  is  actually  deduced  by  introduction  of  implication  rule  (see  the  proof 
below),  so  modus  ponens  is  not  allowed.  Of  course,  we  could  try  to  normalize 
successive  subinferences  of  M(l),  M(2),  etc.  by  hand  or  by  computer.  However, 
this  enterprise  will  be  successful  only  for  some  initial  part  of  this  sequence. 
Surely,  even  M(50)  will  be  never  “normally”  proved  (with  any  abbreviations  for 
formulas  and  proofs  but  not  for  terms). 

Proof  of  Fact  4.  The  cases  of  5(0)  and  M(0)  are  trivial.  ->5(10)  and  -iM(1024) 
easily  follow  from  the  axiom  Vi/(log2  log2  3/  <  10).  The  proof  of  Vx(M(x) 
M(x  +  1))  uses  the  axiom  on  log2  by  inferring  first  M(x  +  1)  from  JVf(x): 

[y  ^  O&x  <  log2  y]  y^  O&x  <  log2  2/  =>  a;  +  1  <  \og2(y  +  v) 


44 


Other  cases  are  also  easy  except  that  of  3x(5'(x)&-i5(a;  -f  1)).  To  prove  this 
we  first  write  down  the  following  natural  deduction  for  several  feasible  numerals 
n  =  0  +  l  +  l  +  ...  +  l 

5(n)  h5(n  +  l)]i 
[-»3a;(5(a;)&:-i5'(a;  +  1))]^  3x{S{x)k~>S(x  +  1)) 

- — -  1 

S{n  +  1) 

Using  successively  these  inferences  for  n  =  0, 1, . . . ,  8,  together  with  the  evident 
proof  of  5(0),  and  the  natural  deduction 


V^(log2  loga  y  <  10) 
[10  <  loga  loga  2/]^  loga  loga  y  <  10 


[5(10)]^ 

X 

e\ 

^(O)  -.5(10) 

3x(5(a;)&-i5(a;  +  1)) 

[-.3a:(5(a;)&-.5(x  -I- 1))]^ 

X 

A 

3a;(5(a;)&:-i5(a;  +  1)) 


we  obtain  the  required  normal  proof  with  feasible  term  size.  □ 

Analogously,  FEAS  3x  6  SWy  E  S{y  <  x).  Note,  that  we  can  prove. 

0g5,  l€5,  ...,465  where  e.g.  the  last  statement  means  that  2^^  =  2^®  <  10^ 
is  a  finite  number.  Much  more  difficult  (if  possible  at  all)  it  is  to  prove  5  6  5 
because  2^^  =  2^^  is  extremelv  large  number.  But  7  and  even  6  are  surely 

not  in  5  because  2^  =2^^®  and  2^  =  2^^  =  16^®  >  10^®  are  intuitively  non- 
feasible  (however  less  than  2^®^^).  It  follows  that  intuitively  the  largest  number 
in  5  is  something  like  4,  or  5  and  that  2^  =  16  6  M  but  2®  =  64  ^  M.  These 
considerations  also  show  that  we  could  strengthen  the  main  axiom  of  theory 
FEAS  as  V2/(loga  loga  2/  <  6)  or  even  as  V2/(loga  loga  V  <  5).  Such  or  other  strong 
versions  of  feasibility  axiom  may  depend  on  computer  technology  available  today. 
However,  we  believe  that  the  truth  value  (or,  better  to  say,  the  role)  of  original 
feasibility  axiom  V2/(loga  loga  2/  <  10)  does  not  depend  on  any  technology. 

We  see  that  5  and  M  have  somewhat  different  properties.  But  the  following 
fact  shows  that  the  story  is  much  more  intriguing.  Simultaneously  with  the 
formula  Vx(M(x)  M{x  +  1))  we  also  have  provable  its  negation: 

Fact  5.  FEAS  -^^x{M{x)  ^  M{x  +  1)). 

Proof.  The  following  deduction  of  X  in  the  theory  FEAS  +  Vx(M(a;)  ^  M{x-\- 
1))  is  normal  (despite  many  applications  of  modus  ponens  which  are  evidently 
allowed  for  partial  cases  of  the  hypothesis  M{x)  M{x  1)): 

{M(0),  (M(0)  =>  M(l)),  M(l),. . . ,  M(1023),  (Af(1023)  =>  M(1024)), 
M(1024)  :=  32/(1024  <  loga  2/),  so,  let  1024  <  loga  2/,  ^ut  this  contra¬ 
dicts  to  the  axiom  V2/(loga  loga  V  <  10),  i.e.  1  is  proved}. 
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Hence,  the  rule  I)  gives  FEAS  M{x  4- 1)).  □ 

We  see  that  Facts  4  and  5  give  FEAS  hj  A  and  FEAS  ~^A,  for  A  := 
\fx{M{x)  =>  M{x  +  1)).  But  this  is  not  a  contradiction  (cf.  Fact  1).  It  follows 
only  that  the  rule  A,  -iA/±  is  not  always  ad.missible,  as  well  as  the  more  general 
modus  ponens  (or  cut)  rule  A,A^  BfB  (because  ~>A  is  A  JL).  So,  we  should 
reconsider  the  question: 


What  is  a  Contradiction? 

We  adopt  here  the  reasonable  convention  that  a  theory  may  be  considered  con¬ 
tradictory  only  if  it  is  trivial,  e.g.  if  all  its  well  formed  formulas  (of  feasible  term 
size)  are  provable  in  our  sense.  Of  course,  FEAS  is  not  such  one.  Nevertheless, 
the  above  unusual  peculiarity  of  FEAS  properly  reflects  the  contradictory  nature 
of  such  fuzzy  notions  as  feasible  and  middle  natural  numbers.  It  is  intuitively 
plausible  that  there  exists  no  maximal  middle  number  (e.g.  the  last  month  of 
our  childhood)  and,  on  the  contrary,  it  is  very  strange  to  think  that  before  1000 
there  exists  an  “inflnite”  increasing  sequence  of  natural  numbers. 

The  reader  may  remember  also  the  related  example  of  a  picture  on  a 
computer  display  which  looks  simultaneously  as  continuous  and  discrete, 
and  the  reason  for  that  is  evidently  just  our  mind,  not  an  optical  effect. 

The  same  holds  for  the  physical  continuum,  because  real  numbers  used 
in  Physics  have  about  30  decimal  digits  after  the  point  or,  equivalently, 
about  100  (the  intuitively  non-middle  number)  of  binary  digits.  So,  there 
is  the  possibility  to  formalize  real  numbers  as  infinite  sequences  of  binary 
digits  so  that  each  digit  after  the  point  will  have  the  number  less  than 
100  (or  1000).  And  the  resulting  continuum  will  be  both  continuous  and 
discrete.  Probably  these  finitary/infinitary  ways  of  arguing  could  have 
some  interesting  effects  if  used  simultaneously  or  mixed  in  one  and  the 
same  proof.  Also  physical  elementary  particles  which  are  considered  both 
as  particles  and  as  waves  may  have  some  releavance  to  feasible  numbers. 


5  A  More  Liberal  Approach 

Let  NK  denote  the  classical  calculus  of  natural  deduction  in  the  form  of  (Prawitz 
1965).  We  will  define  below  another  restricted  version  NK^  of  the  calculus  NK 
whose  inferences  do  not  use  term  abbreviations  in  some  more  natural  and  not  so 
restrictive  sense  than  it  was  considered  above.  Deducibility  in  these  calculi  will 
be  denoted,  respectively,  as  T  l-°  A  and  F  A  where  F  \-^  A  evidently  must 
imply  F  A.  The  converse  implication  T  h  A  implies  T  h®  A’  also  will  hold 
(classically),  however,  h®-deduction  could  be  of  too  large  size  than  the  initial 
F-deduction.  Especially  crucial  for  us  is  a  possible  lengthening  the  size  of  the 
participating  terms  due  to  using  their  abbreviations  in  the  initial  deduction. 


46 


Let,  as  above,  the  subscript  ‘f’  denote  deducibility  with  a  (real)  feasible  size  of 
participating  terms.  Then  we  will  have  trivially 

A  implies  r\-f  A 

where  the  converse  implication  will  not  always  hold.  In  particular,  some  theory 
T  may  be  inconsistent  in  the  sense  of  h/  (T  h/  ±),  but  not  in  the  sense  of  hj 
X). 

On  the  other  hand, 

r\-{\-^)A  iff  r\-^A, 

where  ‘n’  symbolizes,  as  above,  the  normality  property  of  natural  deduction. 
The  ‘if’  case  holds  because  h®  will  be  defined  to  generalize  h”.  The  ‘only  if’  case 
may  give  rise  to  a  considerable  increasing  the  whole  size  of  the  deduction,  except 
for  the  term  size,  as  we  will  see  below: 

rh^A  iff  ri-^A  (1) 

This  is  the  reason  why  we  could  freely  use  more  liberal  notion  of  deducibility  hj 
instead  of  hj  for  formalizing  any  feasibility  theory,  like  FEAS.  The  calculus  NK° 
or  its  ramified  version  NK'  defined  below  may  be  more  comfortable  to  work  with 
than  normalized  NK. 

To  define  NK°  let  us  consider  an  auxiliary  calculus  NK'  (with  the  corre¬ 
sponding  deducibility  relation  h')  obtained  from  NK  by  extending  the  first-order 
language  of  NK  by  a  weak  quantifiers  V'  and  3'  and  by  replacing  NK-rules  of 
introduction  for  V  and  3  and  also  the  classical  NK-rule  (±c)  for  the  falsity  by 
the  following  rules 


M] 


Mt) 

^'xA{x) 


VxA{x) 


(V'l) 


where  in  the  rule  (JL^)  the  formula  A  has  not  the  form  ^xB{x)  or  3xB{x) 
(but,  e.g.,  may  have  the  form  YxB{x)  or  3'xB{x)).  In  particular,  quantifier 
elimination  rules  may  be  applied  only  to  strong  quantifiers  V  and  3 


[A{x)] 


3xA{x) 

B 


yxA{x) 


(VE) 


If  V  is  an  inference  in  the  resulting  calculus  NK'  then  denotes  the  result  of 
the  replacement  of  all  occurrences  VxB{x)  and  3' xB{x),  respectively,  by  \/xB{x) 
and  3xB{x).  Then  we  define  NK°-deductions  as  deductions  of  the  form  if  where 
V  is  any  deduction  in  NK'.  We  introduce  analogously  the  denotations  A^  and 
for  arbitrary  formula  A  and  list  of  formulas  T  in  the  extended  language. 
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Note,  that  V  :  F  A  implies  X>®  :  F^  l-°  A^.  We  also  have  that  V  :  F  \-^  A 
iff  D'  :  r'  h'  A'  for  some  D'  such  that  (X>0°  =  V,  (r)°  =  F  and  (A')°  =  A. 

The  calculus  NK®  explicates  (via  NK')  the  requirement  that  no  quantifier 
introduced  will  be  eliminated  in  a  NK®(NK')-derivation.  Therefore,  no  (using  of) 
term  abbreviations  are  allowed  in  NK®  and  this  is  achieved  with  much  more  weak 
restriction  than  normality.  However,  we  have  the  following  connection  with  the 
approach  presented  before  this  section. 

Detour-conversions  (Troelstra  and  van  Dalen  1988)  (except  for  V-  and  3- 
conversions  which  are  impossible  here  due  to  our  splitting  the  quantifiers  ac¬ 
cording  to  their  I-  and  E-rules),  ±c’ conversions^^ ,  permutation  conversions  and 
immediate  simplifications  are  applicable  to  NK'-derivations  as  to  NK-derivations. 
Moreover,  no  new  terms  will  be  introduced  in  the  derivations  during  this  pro¬ 
cess.  It  follows  that  (without  using  quantifier  conversions)  we  may  (potentially) 
normalize  each  NK'-derivation,  as  for  the  case  of  NK,  however,  evidently  without 
introducing  new  terms  and  therefore  with  preserving  the  term  size  of  the  given 
NK'-derivation.  This  gives  the  one  half  of  the  above  mentioned  equivalence  (1). 
The  other  half  follows  from  a  more  general  result  that  not  only  quantifiers,  but 
also  all  logical  connectives  in  a  normal  natural  deduction  may  be  split  into  the 
weak  and  strong  ones  according  to  their  participating  in  the  rules  of  the  de¬ 
duction.  We  postpone  the  corresponding  detailed  considerations  to  some  other 
paper. 

In  the  case  of  NK®  we  have  no  usual  semantics  for  logical  connectives,  e.g., 
for  the  implication  So,  modus  ponens  is  again  not  always  applied  to  =>■.  The 
case  of  NK^  seems  better  for  the  propositional  connectives  (no  restrictions  on 
the  corresponding  rules!).  However  the  intuitive  understanding  the  quantifiers 
requires  some  comments. 

So,  given  a  proof  V{x)  of  A{x)  for  an  arbitrary  a;,  i.e.  essentially  a  proof 
of  VxA{x),  it  may  be  problematic  to  obtain  a  proof  of  A(t)  for  any  term  t 
of  feasible  size.  The  usual  substitution  V{t)  works  badly  because  t  may  have 
several  occurrences  in  V{t)  (or  in  some  other  term  of  X>(t)).  One  separate  such 
substitution  may  be  sufiiciently  harmless,  and  this  could  be  considered  as  a 
justification  of  the  missing  in  NK'  rule  (V'E).  This  makes  corresponding  strong 
and  weak  quantifiers  V  and  V'  “almost”  the  same.  (Evidently,  'ixA{x)  =>>  V'xA(x) 
and  the  converse  implication  was  discussed  just  now  as  “almost”  true.)  However, 
doing  this  repeatedly  may  result  in  trying  to  consider  deductions  with  terms  of 
non-feasible  size. 

Analogous  consideration  is  applicable  to  3  and  3'  (as  well  as  for  the  possible 
analogous  splitting  of  implication  or  disjunction). 

6  Further  Possible  Developments 

To  make  the  theory  of  feasible  numbers  more  appropriate  to  applications  in 
Computer  Science  it  should  be  reformulated  for  more  rich  data  types  than  nat- 

to  be  defined  appropriately;  in  (Prawitz  1965)  and  (Troelstra  and  van  Dalen  1988) 
there  is  no  corresponding  definition. 
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ural  numbers,  for  example,  for  finite  strings  in  some  finite,  e.g.  binary  alphabet 
or  for  hereditarily  finite  sets.  Also  intuitionistic  as  well  as  higher-order  versions 
of  theories  discussed  could  be  considered.  A  good  mathematical  theory  should 
be  sufficiently  rich  to  describe  a  computability  notion  adequately.  For  example, 
in  Bounded  Set  Theory  (Sazonov  1987)  provably  recursive  operations  over  HF- 
sets  coincide  with  polynomial-time  computable  ones.  Fecisible  Set  Theory,  if  any, 
probably  should  have  some  features  of  BST  and  of  Alternative  Set  Theory  of 
P.Vopenka  (1979). 

In  the  case  of  strings,  (feasible)  natural  numbers  are  identified  with  unary 
strings  (those  in  one-letter-alphabet).  Then,  the  addition  operation  “-1-”  is  gen¬ 
eralized  to  the  concatenation  of  feasible  binary  strings.  Some  other  useful  oper¬ 
ations  over  strings  may  be  introduced,  as  well,  with  the  requirement:  The  value 
of  any  (intuitively)  feasible  closed  term  should  be  a  feasible  string. 

Then  binary  strings  of  the  length  1000  (or  100  or  even  64,  because  intuitively 
64  0  M)  may  be  naturally  considered  as  real  numbers  (in  binary  notation) 
between  0.000 . . .  and  1.000  ...  =  0.111  —  They  are  naturally  factorised  modulo 
equivalence  relation  of  approximate  equality 

X  ^  y  :=^i  £  M{xi  =  yi)  V  two  symmetric  disjuncts  of  the  kind 

eM\ii<j {xi  =  yi)kxj  =  O&i/j  =  G  M{i >j  ^  Xi  =  Ikyi  =  0)] 

where  Xi  denotes  the  i-th  digit  of  string  x.  We  may  try  to  develop  Mathemat¬ 
ical  Analysis  (e.g.  to  prove  that  sin' a;  «  cosrc,  etc).  The  advantage  of  such  a 
kind  of  Nonstandard  Analysis  would  be  corresponding  “smooth”  computability 
theory  with  real  numbers  containing  only  bounded  (or  even  suitably  fixed,  e.g. 
100)  number  of  binary  digits.  Note,  that,  in  contrast  to  the  ordinary  Robin¬ 
son’s  approach,  “nonstandard”  methods  seem  inevitable  in  developing  Feasible 
Mathematical  Analysis. 

The  “set”  of  feasible  numbers  F  may  be  naturally  considered  as  a  proper 
initial  part  of  the  set  P  of  polynomial  numbers  (and  strings)  which  is  closed 
not  only  under  addition  “-b”  but  also  under  multiplication  “•”.  It  follows  that 
21000  ^  P\F^  because  2^°^°  =  2'2-2-...*2  and  the  right-hand  side  is  now  a  legal 
term  of  feasible  length.  However,  2^  ^  P  because  there  is  no  feasible  term  in 

the  language  0, 1,  H-,  •  which  denotes  this  “imaginary”  number. 

The  corresponding  theory  POL  of  polynomial  numbers  and  finite  strings  of 
polynomial  length  may  be  formulated  (like  FEAS)  as  POLq  +  the  axiom 

Vi/  G  P(log2  log2  log2  y  <  10) 

which  means  that  =  oo.  Here  POLq  denotes  sufficiently  reach  list  of  closed 
V-formulas  which  are  “true”  over  the  “ordinary”  finite  binary  strings.  The  lan¬ 
guage  of  POL  and  POLq  consists  of  some  finite  list  of  symbols  for  functions  over 
finite  binary  strings  which  are  sufficient  to  express  by  terms  all  polynomial-time 
computable  functions  (cf.  Sazonov  1980a,  1989).  Moreover,  it  is  required  that 
this  language  does  not  contain  the  number  function  x’^.  More  exactly,  we  only 
have  the  multiplication  x  •  y  and,  in  particular,  a;  ■  a;,  where  the  abbreviation  x^ 
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for  the  latter  is  not  allowed  and  the  corresponding  one-place  function  symbol 
does  not  exist  in  the  language.  Otherwise,  we  could  feasibly  denote  the  (imag¬ 
inary)  number  2^  as  (. . .  ((2^)^)^  . .  .)^  and  prove  its  finiteness  and  therefore 
infer  the  contradiction  in  POL. 

We  may  define  in  POL  feasible  numbers  by  the  predicate  F{x)  :=  P{x)Sc3y  G 
P{x  <  log2  y).  Then  middle  and  small  numbers  should  be  redefined,  respectively, 
as  M(x)  :=  P{x)k3y  G  P{x  <  log2log2y)  and  S{x)  :=  P(x)kBy  e  P{x  < 
logg  log2  log2  y).  (Cf.  the  definition  of  the  predicates  in  §  3.) 

As  in  weak  theories,  say,  of  (Sazonov  1980a,  1989),  the  quantifier-free  in¬ 
duction  scheme  is  provable  in  POL.  Also  partial  recursive  functions  via  Turing 
computability  can  be  described,  as  well  as  universal  Turing  machine,  s-m-n  theo¬ 
rem  and  recursion  theorem  (Sazonov  1980a,  1989).  In  particular,  exponentiation 
2®  is  a  partial  recursive  function  here.  It  is  undefined  for  x  =  non-feasible  poly¬ 
nomial  number  but  its  value  on  feasible  number  x  —  1024  is  defined  and 
non-feasible.  Then  provably  recursive  functions  of  POL  (i.e.  those  partial  recur¬ 
sive  functions  whose  totality  can  be  proved  in  POL)  are  just  polynomial- time 
computable  ones  (over  polynomial  binary  strings). 

Note,  that  theory  FEAS,  even  if  suitably  extended  from  numbers  to  strings, 
seems  not  very  appropriate  to  develop  Turing  computability.  Indeed,  multiplica¬ 
tion  missing  in  FEAS  is  necessary  to  estimate  the  time  of  simulating  any  Turing 
machine  by  an  universal  one  and  to  prove  this.  Nevertheless  we  may  try  to  de¬ 
fine  suitably  the  notion  of  partial  recursive  functions  with  the  help  of  some  other 
model  of  computation  in  such  a  theory.  Then  corresponding  provably  recursive 
functions  could  be  naturally  called  linear-time  or  feasibly  computable.  Generally, 
such  a  way  of  arguing  may  be  considered  as  a  method  to  estimate  “naturalness” 
of  various  notions  of  computability  and  complexity  theory. 

The  discussion  in  §  3  shows  that  quantification  (=  term  abbreviation)  rules 
in  non-normal  proofs  give  rise  to  non-feasible  (or  to  non-polynomial,  etc.)  num¬ 
bers.  However,  if  we  know  that  the  value  of  a  (possibly  very  complicated)  term 
is  bounded  by  the  value  of  some  other  term  which  will  be  never  abbreviated  in 
a  proof,  then  the  first  term  may  be  freely  abbreviated  without  any  such  unde¬ 
sirable  non-feasibility  effect.  Hence,  the  normality  requirement  on  proofs  may  be 
somewhat  weakened  as: 

Formula  occurrences  in  a  proof  where  normality  fails  (or  cut 
formulas)  must  contain  only  bounded  quantifiers.  Additionally, 
Bounded  Induction  Axiom  may  be  allowed. 

Here  also  something  interesting  may  appear.  Such  abbreviations  for  finite 
binary  strings  of  a  bounded  length,  may  give  rise  to  not  lengthy  but  very  complex 
strings  (cf.  the  notion  of  Kolmogorov’s  complexity  of  strings  and  the  notion 
of  constructive/ non-const ructive  finite  strings  (Sazonov  1980a,  1989)).  But  we 
might  want  to  consider  only  simple  (i.e.  not  complex)  binary  strings,  as  it  was 
above  with  feasible  and  polynomial  numbers.  So,  we  should  choose  which  notion 
of  binary  string  we  are  interested  in  and  respectively  decide  whether  the  above 
kind  of  abbreviations  for  terms  with  bounded  values  is  allowed  or  not. 
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It  is  very  desirable  to  develop  corresponding  informal  style  of  “Feasible  Math¬ 
ematics  Thinking”  like  “Model-Theoretic  Thinking”  of  classical  mathematics 
which  allows  to  prove  theorems  sufficiently  rigorously,  but  without  using  for¬ 
malized  predicate  calculus.  For  this  aim  we  could  begin  with  considering  more 
and  more  liberal  and  convenient  formalizations  of  corresponding  logic  as  it  was 
attempted  above.  After  all,  a  good  formalizm  is  such  one  which  we  use  without 
too  strong  effort. 

Note,  that  Godel’s  argument  on  non-provability  in  arithmetic  of  its  consis¬ 
tency  does  not  work  for  formalisms  (like  FEAS)  without  modus  ponens  rule. 
Therefore,  it  is  interesting  to  see  what  exactly  will  take  place  in  our  case. 

The  author  sees  no  unsurmountable  obstacle  to  proceed  in  these  directions 
with  all  necessary  technical  details.  However  such  a  work  evidently  must  be  a 
heavy  one  because  any  restrictions  on  logic  and  on  arithmetic  require  some  more 
attention  and  ingenuity  than  usually. 
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Abstract.  This  paper  defines  natural  hierarchies  of  function  and  rela¬ 
tion  classes,  constructed  from  parallel  complexity  classes  in  a  manner 
analogous  to  the  polynomial-time  hierarchy.  A  number  of  structural  re¬ 
sults  about  these  classes  are  proven:  relationships  between  them  and  the 
levels  of  PH,  a  Buss-style  witnessing  theorem  relating  the  levels  of  these 
hierarchies  to  definabihty  in  the  bounded  arithmetic  theories  Rl  (gener¬ 
alizing  [1]  and  improving  on  [9]),  a  conservation  result  between  SI  and 
and  results  analogous  to  those  of  [18,  8,  16]  relating  conservation 
between  theories  of  bounded  arithmetic  to  the  collapse  of  complexity 
classes. 


1  Introduction 

Rohit  Parikh  [21]  studied  weak  theories  of  arithmetic  that  have  induction  only  for 
bounded,  or  Aq,  formulae.  These  theories  became  known  as  theories  of  bounded 
arithmetic^  and  were  further  developed  by  Sam  Buss  in  [6].  Buss  restricted  in¬ 
duction  not  only  to  bounded  formulae,  but  to  bounded  formulae  of  fixed  com¬ 
plexity  Uf.  Furthermore,  he  distinguished  between  an  exponential-length  and  a 
polynomial-length  form  of  induction,  defining  thereby  two  related  hierarchies  of 
logical  theories  T}  and  SI,  and  proved  basic  relationships  among  them  such  as 
that  ^2  C  T2  C  ^2^^  for  i  >  1.  Most  importantly.  Buss  showed  that  the  proof 
strength  of  S2  corresponded  exactly  to  accepted  notions  of  computational  com¬ 
plexity:  the  functions  at  level  i  of  the  polynomial  time  hierarchy  are  precisely 
those  with  a  graph,  provably  total  and  single- valued  in  S^- 

Expanding  on  this  work,  various  researchers  (e.g.  [1,  11,  12])  introduced  a 
third  hierarchy  of  theories  based  on  log-length  induction,  and  demonstrated 
that  Rl  analogously  characterized  the  functions  computable  in  the  parallel  class 
AfC.  But  no  natural  description  was  known  of  the  computational  content  of 
Rl  proofs  for  i  >  1,  in  part  because  many  natural  arguments  about  log-length 
induction  seem  to  require  quasipolynomial,  rather  than  polynomial,  growth  rates 
(although  see  [5]  for  an  exception).  Attention  therefore  turned  to  the  theories 
Rl,  in  which  terms  have  quasipolynomial  growth.  Buss,  Krajicek,  and  Takeuti  [9] 
described  the  problems  for  which  Rl  can  prove  the  existence  of  solutions,  relying 
on  a  somewhat  complex  notion  of  multi-valued  functions  computed  with  a  limited 
number  of  queries  to  a  witnessing  oracle. 

*  This  work  was  sponsored  in  part  by  Judy  Goldsmith’s  NSERC  operating  grant 
OGP0121527  while  the  author  was  at  the  University  of  Manitoba;  further  work  was 
done  at  the  University  of  Kentucky. 
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The  present  paper  characterizes  (see  Theorem  35)  the  functions  provably  to¬ 
tal  and  single-valued  in  jRgj  -RJ}  -RJ,  and  so  on,  without  the  complications  of  multi¬ 
valued  functions,  witnessing  oracles,  or  artificial  bounds  on  the  number  of  oracle 
queries.  These  simpler  characterizations  depend  both  on  the  function-theoretic 
characterization  of  [4]  and  on  circuits  with  oracle  gates,  as  in  Wilson  [25,  26]. 
But  where  Wilson  constructed  oracles  at  recursive  and  higher  levels  to  prove 
relativized  complexity  results  among  classical  complexity  classes  like  A/‘'P,  the 
present  paper  uses  oracles  within  classical  complexity  classes  to  construct  and 
describe  new  classes  analogous  to  those  of  the  polynomial  hierarchy. 

We  answer  a  question  posed  in  [9]  by  proving  that  if  R\  h  iyx){3y)(j){x^y), 
for  <j)  G  then  there  is  a  function,  provably  total  and  single-valued  in  that 
witnesses  the  statement.  (See  Theorems  41  and  44.)  We  also  give,  in  Theorem  47, 
a  simpler  proof  of  the  conservation  result  in  [9].  That  paper  also  asked  whether 
the  known  conservativity  between  and  Rl'^^  also  holds  between 

and  R^^^ ;  we  give  circumstantial  evidence  that  it  does  not,  as  such  conservativity 
would  imply  a  collapse  of  complexity  classes  (see  Theorem  49). 

Knowing  that  5^  characterizes  level  i  of  the  polynomial  time  hierarchy,  or 
that  R\  characterizes  level  i  of  a  natural  parallel  hierarchy,  does  not  in  itself  an¬ 
swer  questions  about  the  collapse  or  non-collapse  of  complexity  classes.  Krajicek, 
Pudlak,  and  Takeuti  [18]  showed  that  if  =  Tj,  then  C  Zif^^/poly  and 

therefore  the  polynomial  hierarchy  collapses  to  at  most  =  ^f+2*  ®^ss  [8] 

simplified  the  proof  and  strengthened  the  result:  under  the  same  hypotheses, 
C  zl^^j/poly,  and  therefore  the  polynomial  hierarchy  collapses,  provably 
in  T2.  The  present  paper  shows  (see  Theorem  51)  that  any  equivalence  of  the 
form  R^^^  =  SI,  with  i  >  1,  fc  >  3,  would  imply  a  similar  collapse  of  complexity 
classes,  e.g.  (for  k  —  S)  the  collapse  of  the  quasipolynomial  hierarchy,  provably 
in  Si 

Another  recent  result,  due  to  Krajicek  [16],  shows  that  if  S^  =  Tj,  then 
=  V^^O{\og{n))]  =  LogSpace^l  We  have  extended  Krajicek’s  result  in 
several  ways:  any  equivalence  S^  =  Tl  or  =  5|,  with  z  >  1,  A:  >  2,  would 
imply  that  the  sequential  function  class  can  be  parallelized  to  run  exponen¬ 
tially  faster;  see  Corollaries  34,45,  and  46. 

Section  2  defines  models  of  parallel  computation,  and  the  complexity  classes 
of  the  parallel  hierarchies,  used  in  this  paper.  Section  3  proves  a  variety  of  basic 
relationships  among  these  classes.  Section  4  shows  a  direct  link  between  these 
classes  and  those  defined  in  [9,  16].  In  Sect.  5  is  a  Buss-style  witnessing  theorem 
to  show  that  the  classes  ^  correspond  precisely  to  the  E\  consequences  of 
Rl  this  implies  an  alternate  proof  of  the  link  shown  in  Sect.  4,  and  an  alternate 
proof  of  the  conservativity  result  of  [9].  Section  6  gives  results  analogous  to  those 
of  [18,  8]  relating  the  collapse  of  theories  of  bounded  arithmetic  to  the  collapse 
of  complexity  classes.  We  conclude  by  discussing  some  of  the  remaining  results 
we  would  most  like  to  prove  about  these  theories. 

The  present  paper  is  intentionally  informal  in  places,  omitting  the  least  in¬ 
teresting  proofs  and  definitions.  A  version  with  these  details  intact  has  been 
submitted  for  journal  publication. 
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2  Definitions 

2.1  Languages,  Notation,  and  Growth  Rates 

We  work  with  several  first-order  languages  of  arithmetic,  similar  to  those  used 
in  [6,  9],  with  whose  notation  I  assume  the  reader  is  familiar.  First  we  define  an 
hierarchy  of  term  languages,  essentially  the  same  as  that  in  [22]. 

Definition  1.  The  language  C\  consists  of 

-  the  variable  symbols  a,  6,  c,  d,  w,  z,  j/,  possibly  subscripted  with  an  integer, 

-  the  constant  symbols  0  and  1, 

-  the  binary  infix  relation  symbols  =  and  <, 

-  the  binary  infix  function  symbols  -t-  and  •,  and 

-  the  unary  outfix  function  symbol  |  •  •  •  |, 

Definition  2.  For  A;  >  1,  the  language  Ck  is  the  language  Ci  augmented  with 
the  binary  infix  function  symbols  #2)  #3?  #4?  •  •  •  #/kj  pronounced  “smash  two,” 
“smash  three,”  and  so  on.  “Smash  two”  is  often  simply  called  “smash” . 

These  symbols  are  intended  to  be  interpreted  as  follows:  z#2y  =  —  1, 

xH^zy  —  2l®1^2|y|  -  j  -  1.  Smash  functions  are  discussed 

in  [13,  15,  20,  6,  1],  all  of  which  use  unimportantly  different  definitions  (for 
example,  omitting  the  “—1”). 

We  shall  frequently  refer  to  “the  length  of  an  >Cfe-term,”  or  \Ck\  for  short. 
For  example,  if  we  say  “/(z)  6  or  “/(z)  is  bounded  by  the  length  of  an  Ci 
term,”  we  mean  that  /  is  at  most  a  linear  function  in  the  length  of  z.  Similarly, 
1^2 1  means  a  polynomial,  or  in  the  lengths  of  the  free  variables;  is 

quasipoly nomiaJ,  or  in  the  lengths  of  the  free  variables,  and  so  on. 

We  shall  likewise  refer  to  “the  log  of  the  length  of  an  /Ife-term,”  abbreviating  it 
\\Ck\\  (the  slight  abuse  of  notation,  equating  log  with  length,  doesn’t  affect  the 
results). 


2.2  Syntactic  Complexity 

We  next  define  an  hierarchy  of  syntactic  formula  classes,  analogous  to  the  classes 
Eiy  Ai  familiar  to  recursion  theorists.  The  following  definitions  differ  from 
Buss’s  in  [6]  mainly  in  the  presence  of  a  second  subscript  k;  since  Buss  was  only 
concerned  with  a  language  analogous  to  £2,  his  results  apply  directly  to  our 
classes  with  A;  =  2. 

Definition  3.  A  bounded  quantifier  with  bound  of  the  form  |t(z)|,  i.e.  one  whose 
outermost  function  symbol  is  |  •  •  •  | ,  is  said  to  be  sharply  bounded. 

Definition  4.  The  classes  JC?  ^  and  are  the  smallest  sets  of  formulae  satis¬ 
fying  the  following  inductive  definition  (where  t  is  a  term  in  language  £jt). 
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-^L  =  Kk  is  the  set  of  formulae  in  language  Ck  in  which  all  quantifiers  are 
sharply  bounded. 

-  nlk  c 

-  If  i4  6  then  the  formula  (3x  <  t)A  is  also  in 

-  li  A  €  then  the  formula  (Vx  <  t)A  is  also  in 

-  If  A,  B  G  Hf  f.  (respectively  then  so  are  A  A  B  and  A  V  B. 

-  If  A  G  Si  f:  (respectively  then  -lA  G  Ilf  (respectively 

-  If  A  G  I^f  k  (respectively  nf  k)^  then  so  are  both  (Vx  <  |t|)A  and  (3x  <  |t|)A. 

When  the  language  Ck  is  clear  from  context,  particularly  when  A;  =  2,  we 
omit  the  subscript  k. 


2.3  Theories  of  Bounded  Arithmetic 

The  theory  Tj,  is  axiomatized  by  a  fixed,  finite  set  of  quantifier-free  axioms  and 
the  inference  rule  Ef  j^  —  IND: 

r(b),A(a,b)  — >  A(a  +  l,b),A(b) 
r(b),A(0,b)^A(t(b),b),Zi(b)  ’ 


where  A  G  Sf  /^  and  t  ^  Ck- 

The  theory  S\  is  axiomatized  similarly,  but  with  IND  replaced  by 
Ef^  -  FIND  (“prefix  induction”): 

r(b),^(|$|,b)^A(a,b),^(b) 
r(b),A(0,b)^A(t(b),b),zi(b)  * 


or  an  equivalent  scheme  named  LIND^  proven  equivalent  in  [6,  1]. 

The  theory  is  axiomatized  similarly,  with  one  of  four  induction  schemes, 
called  DC/,  PPIND,  LPIND,  and  LLIND.  They  are  shown  to  be  equivalent 
in  [1,  12]  (in  a  sufficiently  expressive  language,  such  as  Ck  for  A;  >  2).  The 
functions  Fh  and  Bh  return  the  “front  half”  and  “back  half”  respectively  of  a 
bit  string. 

Ef  f^  “  BCJ  (“divide  and  conquer  induction”): 

r(b),  A(B/i(o),b),  A(F/i(a),b)  — ►  A(a,b),^(b) 
r(b),  A(0,b),  A(l,b),  A(2,b),  A(3,b)  ^  A(t(b),b),  A(b)  * 

The  reader  seeking  more  detail  and  discussion  of  these  theories  and  induction 
schemes  is  referred  to  [6,  1,  12]. 


56 


2.4  Circuits  and  Uniformity 

Our  complexity  classes  are  defined  in  terms  of  uniform  families  of  multiple- 
output  circuits,  which  furthermore  may  contain  “oracle  gates”.  Wilson  [25,  26] 
introduced  a  similar  notion,  and  I  follow  his  convention  that  an  oracle  gate’s 
“size”  and  “depth”  are  its  fanin  and  the  log  of  its  fanin  respectively. 

Most  the  circuit  families  in  this  paper  are  described  by  genus: 

Definition  5,  An  oracle  circuit  family  of  genus  fc  is  a  circuit  family  with  at  most 
\Ck\  output  gates,  depth  at  most  UAH?  and  fanin  at  most  2  except  for  its  oracle 
gates.  All  oracle  gates  in  a  given  circuit  family  with  input  x  have  fanin  exactly 
|t(x)|,  where  t  is  a  nontrivial  term  in  Ck- 

(By  “nontrivial”  we  mean  “not  much  smaller  than  any  other  term  in  £fc”:  bound¬ 
ing  t  below  by  the  maximum  of  its  free  variables  suffices.) 

Most  results  in  this  paper  are  stated  “for  k  >  2”  or  “for  >  3”.  (Wilson’s 
convention  on  the  size  and  depth  of  oracle  circuits  makes  no  difference  for  k  >  3.) 
Some  would  hold  even  for  A;  =  1,  but  genus  1  circuit  families  are  so  ill-behaved 
(e.g.,  not  closed  under  composition)  that  we  shall  ignore  the  k  =  1  case. 

Fact  6.  For  k>2,  an  oracle  circuit  family  of  genus  k  has  size  (i.e.,  number  of 
gates)  at  most  \Ck\- 

All  circuit  families  in  this  paper  are  assumed  to  be  uniform  in  the  Ue*  sense 
of  Ruzzo  [23].  However,  we  slightly  extend  Ruzzo’s  definition  of  the  extended 
connection  language  to  handle  multiple-output  circuits  and  oracle  gates.  The 
details  appear  in  the  journal  version  of  this  paper. 


2.5  Complexity  Classes 

The  polynomial-time  hierarchy  contains  relation  classes  Uf,  Ilf,  and  Af,  and 
function  classes  Of  (see,  e.g.  [6]).  Buss  mentions  in  passing  that  analogous  def¬ 
initions  and  theorems  hold  if  the  language  is  expanded  to  Ck,  for  k  >  2.  We 
therefore  generalize  the  notation: 

Definition 7.  The  complexity  classes  Uf  f,,  Tlff^,  Af  f^,  and  Of  ^^,  ioi  k  >  1,  are 
defined  as  follows: 

-  =  Af  j^  =  V. 

-  For  i  >  0,  Sf^i  k  is  the  closure  of  77?^  under  jC^-bounded  existential  quan¬ 
tification. 

-  For  i  >  0,  is  the  closure  of  i7f  ^  under  -bounded  universal  quan¬ 

tification. 

-  For  i  >  0. 

-  For  i  >  0,  =  PP^S’.*. 
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For  example,  AI2  is  V,  Z'f  3  is  ^f'P,  and  is  the  class  of  functions  com¬ 
putable  in  quasipolynomial  time  with  oracles  for  nondeterministic  quasipoly¬ 
nomial  time.  We  also  define  analogous  classes  based  on  parallel  complexity. 

Definitions.  The  complexity  classes  X’f  fe,  ^  ^  1?  stre 

defined  as  follows: 

-  X!q  =  IIq  =  Aq  =  the  class  of  relations  decidable  by  a  uniform  family 
of  single-output  circuits  of  genus  A;,  containing  no  oracle  gates. 

-  For  i  >  0,  is  the  closure  of  under  £jfe-bounded  universal  quan¬ 

tification. 

-  For  i  >  0,  A^_^_l  f,  is  the  class  of  relations  decidable  by  a  uniform  family  of 
single-output  circuits  of  genus  k,  containing  oracle  gates  for  some  or 
77?^  relation. 

-  jt  is  the  class  of  functions  computable  by  a  uniform  family  of  |£fc|-output 
circuits  of  genus  k,  with  no  oracle  gates. 

-  For  i  >  0,  is  the  class  of  functions  computable  by  a  uniform  family 

of  |£fc|-output  circuits  of  genus  A:,  containing  oracle  gates  for  some  or 
Ilf  relation. 

Henceforth  families  with  the  uniformity,  size,  depth,  and  gate  restrictions 
above  will  be  called  Uff,,  Ilf  f,,  and  Af  j^  circuit  families  respectively. 

The  main  theorem  of  Sect.  5  doesn’t  appear  to  hold  in  its  most  elegant  form 
for  A;  =  2  —  indeed,  if  it  did,  it  would  imply  AfC  =  AfC^.  However,  we  can  prove 
a  less  elegant  analogue  for  complexity  classes  defined  in  terms  of  AfC^  with  the 
depth  bounds  of  and  the  size  and  fanin  bounds  of  □§^2- 

Definition 9  (Allen).  TMC  comprises  the  functions  computable  by  Ue*- 
uniform^  circuit  families  with  polylog  depth,  polynomial  size,  and  polynomially 
many  outputs. 

Definition  10.  For  any  class  X  of  relations  on  {0, 1}*, 

-  is  the  class  of  functions  computable  by  -uniform,  polylog  depth, 
polynomial  size,  oracle  circuit  families  with  oracle  gates  of  fanin  |t|  (where 
t  €  £2)  for  an  X  relation. 

-  MC^  is  the  class  of  relations  whose  characteristic  functions  are  0/1- valued 

functions. 

As  usual,  an  oracle  gate’s  “depth”  and  “size”  are  considered  to  be  ||t|l  and  \t\ 
respectively. 

Normally  we  shall  study  classes  of  the  form  and  ^  by 

analogy  to  Af^^  and  Df 

^  Allen  actually  defined  circuit  families  to  be  logspace  uniform,  but  as  he  points 
out,  it  makes  no  difference  at  this  level; see  [23]. 
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2.6  Alternating  Turing  Machines 

We  can  also  define  an  alternating  Turing  machine  model  to  correspond  to  arbi¬ 
trary  levels  in  the  hierarchies. 

Definition  11.  An  oracle  alternating  Taring  machine  of  genus  k  is  an  alternat¬ 
ing  ||£/5;||-time  Turing  machine  whose  states  are  partitioned  into  universal,  exis¬ 
tential,  exclusive-or,  and  oracle  states.  The  universal,  existential,  and  exclusive- 
or  states  each  have  exactly  two  next  states.  The  behaviour  of  all  oracle  states  in 
a  given  machine  depends  on  a  global  “oracle  bound”  t  E  Ck,  “nontrivial”  in  the 
same  sense  as  defined  in  Sect.  2.4. 

When  the  machine  enters  an  oracle  state  for  relation  p,  it  guesses  a  binary 
number  0  <  j  <  |t(a;)|,  writes  it  and  a  delimiter  at  the  end  of  a  special  oracle  tape, 
and  goes  on  to  a  next  state  deterministically.  Let  Dj  denote  the  accept/reject 
decision  of  this  next  state  given  the  number  then  the  oracle  state  accepts  iff 
the  relation  p  holds  of  the  |t|-bit  string  Dit\-iD^t\~2  *  *  •  D2D1D0.  An  oracle  state 
is  counted  as  taking  ||t||  time  steps. 

(Recall  that  Chandra,  Kozen,  &  Stockmeyer  [10]  defined  ATM’s  with  NOT 
states,  then  proved  that  they  could  be  eliminated.  This  proof  appears  not  to 
work  in  the  presence  of  oracle  states,  unless  the  oracles  come  from  a  class  closed 
under  complement.  We  use  two-input  exclusive-or  states  instead,  as  they  can  be 
“programmed”  to  compute  either  a  NOT  or  an  identity  function  by  making  one 
of  their  successor  configurations  unconditionally  accept  or  reject.) 

For  example,  if  p  were  a  |i|-way  AND,  a  p-oracle  state  would  be  equivalent  to 
universally  guessing  ||t||  bits,  which  could  have  equally  well  been  done  without 
an  oracle  in  the  same  time.  Similar  reasoning  holds  for  any  p  E  thus  we 
have  the 

Lemma  12.  For  k  >2,  a  relation  is  computable  by  an  oracle  ATM  of  genus  k 
with  a  A Q  oracle  iff  it  is  computable  by  an  ATM  of  genus  k  with  no  oracles  at 
all 


A  more  interesting  situation  arises  when  p  is,  say,  a  17^  ^  relation;  then  there 
is  no  obvious  way  to  simulate  the  oracle  in  less  than  \Ck\  parallel  time.  The 
following  lemma,  whose  proof  appears  in  the  journal  version,  may  be  seen  as  a 
generalization  of  Ruzzo’s  theorems  3  and  4. 

Lemma  13.  For  fc  >  2,  a  relation  is  computable  by  an  ATM  of  genus  k  with  an 
oracle  in  k  ^ff  l^he  relation  is  in  Ajj^. 

In  particular,  A^  f^  is  equal  to  ATIME{\\Ck\\)  for  all  k>2.  For  example,  Zig  2 
is  known  in  the  literature  as  ALOGTIME  or  uniform  and  Zig  3  is  the  class 
of  problems  solvable  in  uniform  polylog  depth,  or  alternating  polylog  time,  or 
deterministic  polylog  space  (for  other  relevant  characterizations  of  these  classes, 
see  [4]). 
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3  Basic  Relationships  among  the  Classes 

In  this  section  we  state  “the  easy  theorems”  about  the  classes  defined  in  the 
previous  section,  showing  they  constitute  a  well-behaved  complexity  hierarchy, 
before  proceeding  to  more  profound  results.  Again,  the  proofs  (all  quite  straight¬ 
forward)  appear  in  the  journal  version  of  this  paper. 

Lemma  14.  For  k>2,  C 

Theorem  15.  For  i  >  l,k  >2,  and 

This  follows  immediately  from  which  in  other  notation  has 

been  known  at  least  since  1974  [14].  In  light  of  this  theorem,  we  can  dispense 
with  proving  basic  properties  of  the  Ef  f^  and  classes:  either  we  already 
know  them,  or  we  are  unlikely  to  be  able  to  prove  them  with  present  techniques. 
Indeed,  we  can  dispense  with  the  notations  Ef  f^  and  altogether.  But  we 
still  don’t  know  much  about  the  Aj  f,  and  ^  classes.  So  we’ll  state  a  number 
of  useful  results  about  these  classes  and  the  relationships  among  them. 

Lemma  16. 

-  The  A\  relations  are  the  relations  with  characteristic  functions  in  . 

-  For  i  >  0,  k  >  If  A^i^  is  closed  under  negation,  finite  disjunction  and 
conjunction,  and  definition  by  A^  cases  ( and  similarly  with  in  place 

-  For  alii'  >i>0  and  k'  >  k  >1,  we  have  A\j^C  A\,  and  Q  Df/  k> . 

Lemma  17.  Fori  >  0  and  k>2,  and  are  closed  under  composition. 

Corollary  18.  For  i  >  0  and  k  >2,  A\^  is  closed  under  substitution  by  DJ 
functions  (and  similarly  with  and  replacing  A^j^  and  Df  respec¬ 

tively). 

Lemma  19.  For  k>2,  A^  y,  =  A\  f,.  (Similarly,  NC  =  .) 

The  proof  of  this  lemma  for  /?  =  2  depends  on  the  convention  that  an  oracle  gate 
with  fanin  |t|  is  counted  as  having  “depth”  ||t||.  The  proof  for  MC  depends  on 
the  convention  that  an  oracle  gate  is  counted  as  having  “size”  \t\. 

Lemma  20.  For  any  i>0,  k>  2,  the  function  /(x)  is  in  iff  bit-graph, 
the  relation  Aj,x.Bit(j, /(x)),  is  in  Ajy^. 

Similarly,  a  function  is  in  iff  its  bit-graph  is  in  . 

Lemma21.  Fori  >  0  and  k  >2,  if  p{x)  is  equivalent  to  {By  <  |s(x)|)cr(x,t/) 
or  to  {iy  <  |s(x)|)cr(x,y),  where  a  €  Ajy.  and  s  €  Ck>  (respectively  a  G  AfC^ , 
with  s  £  C2),  then  p  is  itself  in  A\  j^  (respectively  ). 

That  is,  A^  y.  and  fifC^  are  is  closed  under  sharply-bounded  quantifiers. 
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Lemma 22.  Fori  >  0,A:  >  2,  C  (Similarly,  C  A^2-) 

Corollary  23.  For  i>0,k>2,  A^  f,  C  0  nf  f., 

(Similarly,  C  r?2  D  nf^.) 

Corollary  24.  For  i  >  ^,  k  >  2,  and  Ilf  f^  are  closed  under  definition  by 
Al^^  and  cases. 

Lemma  25.  For  i>l,k>2,  C  and  C 

Corollary  26.  For  i>^,k>2,  and  are  closed  under  substitution  by 
^i,k  )  functions. 

Theorem 27.  For  i  >  l,k  >  2  we  have  the  inclusions  Aj^^  C  A^f^  C  A^^^  f, 
<^ndni,cnl^cno^^^,. 

4  The  Buss-Krajfcek-Takeuti  Characterization 

Buss,  Krajicek,  and  Takeuti  [9]  give  a  characterization  of  the  X'I’  g-definable  func¬ 
tions  of  jRg.  The  computational  model  in  that  paper  uses  “witnessing  oracles”, 
which  answer  an  existential  query  either  with  “no”  or  by  providing  a  satisfy¬ 
ing  value  (of  quasipolynomial  length)  for  the  outermost  existential  quantifier. 
Since  there  may  be  multiple  possible  witnesses  to  a  given  existential  question, 
the  functions  defined  by  invoking  these  witnessing  oracles  may  be  multi-valued. 
Nevertheless,  in  certain  cases  this  computational  model  is  equivalent  to  the  par¬ 
allel  model  of  the  present  paper. 

The  main  result  of  this  section.  Theorem  31,  is  a  direct  proof  that  the  function 

class  FV^^~^[wit,\og^^^^  of  Buss,  Krajicek,  and  Takeuti,  restricted  to  single¬ 
valued  functions,  is  precisely  DJg.  The  same  result  could  also  be  proven  from 
Theorem  35  together  with  results  of  [9]. 

Definition  28  (Buss,  Krajicek,  Takeuti).  A  multi-valued  function  /  is  com¬ 
putable  in  [mt,log^^^^]  iff  there  is  a  Turing  machine  that  runs  in  quaisi- 

polynomial  time  and  makes  at  most  polylog  many  queries  to  a  fixed  wit¬ 
nessing  oracle,  such  that  for  all  x,  M  outputs  one  of  the  values  of  /(x). 

The  notation  TV^  ,  without  the  stuff  in  brackets,  is  what  I  call  Note  that 
the  oracle  may  (in  my  notation)  be  ^  for  any  k  <3,  without  changing  the 
class  thus  defined:  the  machine  making  the  query  can  precompute  the  quantifier 
bounds  and  pass  them  to  the  oracle  as  extra  parameters. 

Definition 29  (Buss,  Krajicek,  Takeuti).  A  multi-valued  function  /  is 
defined  by  theory  T  iff  for  some  formula  A, 
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-  T  h  {\/x){3y)A{x,y),  and 

—  if  A{n,m)  holds  in  the  natural  numbers,  then  m  is  a  value  of  f{n). 

(Note  that  the  second  requirement  does  not  say  “if  and  only  if’.  Buss,  Krajicek, 

and  Takeuti  also  defined  a  class  called  “strong  log^^^^]”  and  a  notion 

of  “strong  Z']’- definability”  using  “if  and  only  if”,  but  the  difference  is  only 
relevant  for  multi-valued  functions  and  so  we  shall  ignore  it  henceforth.) 

Theorem  30  (Buss,  Krajicek,  Takeuti).  Fori  >  2,  a  multi-valued  function 
f  is  in  F'P^*~^'^[wit,\og^^^^]  iff  it  is  Ef  ^-defined  by  theory  R^. 

TheoremSl.  Fori  >  2  and  k  >  3,  the  restriction  of  FVf^*~^'^[wit,\\Ck\W  to 
single-valued  functions  is  . 

Proof.  (D):  Suppose  /  €  Then  /  is  coniputable  by  a  uniform  Df  circuit 
family  with  yes/no  oracle  p. 

With  at  most  a  constant  factor  increase  in  depth,  we  can  assume  the  circuit 
family  is  levelled,  that  all  output  gates  are  at  level  0,  that  each  even  level  contains 
no  oracle  gates,  and  that  each  odd  level  consists  entirely  of  oracle  gates.  (The 
last  restriction  requires  excluding  const  ant- valued  p.)  Embed  the  resulting  gates 
in  a  rectangular  array,  indexed  by  row  and  column  (r,  c),  in  such  a  manner  that 
a  gate’s  position  in  the  array  easily  determines  its  gate  number  and  thence  its 
type.  Thus  the  resulting  circuit  family  is  still  Ue*  uniform. 

The  output  of  gate  (r,c)  can  be  described  by  a  relation  a-(n,r,  c,b) 

where  n  is  the  input  size  and  b  are  the  outputs  of  row  r  +  1.  (The  relation  a 
simply  determines  the  type  of  gate  (r,  c)  and  simulates  it;  the  most  complex  case 
is  that  of  an  oracle  gate,  which  is  Define  </>(7i,r,b,  j)  to  be  the  formula 

(3\w\  =  width  of  row  r)(|u;|i  >  i)  A  (Vc  <  \w\){Bit{c,w)  D  o-(|x|,r,  c,  b)))  , 

where  |iu|i  represents  the  number  of  I’s  in  w.  The  (f)  relation  is  f,  (note 
i  >  2),  and  monotone  in  j,  so  for  any  fixed  x,  we  can  find  the  maximum  j 
satisfying  0  by  binary  search  in  |  ||  time.  For  this  maximum  j,  the  only  possible 

witness  w  is  the  string  representing  exactly  the  values  of  all  the  gates  at  level  r 
(assuming  b  correctly  represents  the  values  of  all  the  gates  at  level  r-j-1),  because 
the  clause  (Vc  <  \w\){Bit(c,u))  D  o-(|x|,r, c, b))  ensures  that  all  the  gates  whose 
bit  in  in  is  1  are  correctly  computed,  and  if  any  gate  whose  bit  in  in  is  0  were 
incorrectly  computed,  j  wouldn’t  be  maximal. 

Thus  ||jCfc||  (^queries,  the  last  one  witnessing,  suffice  to  determine  the  outputs 
of  all  gates  at  level  r  from  the  outputs  of  all  gates  at  level  r  +  1.  Iterating  this 
over  the  ||>Cfc||  levels,  starting  with  the  input,  allows  us  to  compute  the  correct 
values  of  all  the  gates  at  the  output  level,  and  thus  the  value  of  the  function. 

(C):  Suppose  /  G  TV^,  [mzt,  1|>Ca;||]  and  /  is  single-valued.  As  shown  in  [9], 
we  can  assume  without  loss  of  generality  that  the  quasipolynomial- time  machine 
only  uses  an  ordinary  yes/no  oracle  until  its  last  query,  at  which  point  it  demands 
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a  witness.  Furthermore,  we  can  assume  without  loss  of  generality  that  it  doesn’t 
demand  this  witness  until  it  has  confirmed  by  yes/no  query  that  there  is  one. 

Let  formula  'tp(x^w)  assert  that  there  is  a  computation  of  the  TVk  machine 
(up  to  the  time  that  it  demands  a  witness)  in  which 

—  the  computation  starts  with  x  on  the  input, 

-  for  all  j,  if  the  j-th  most  significant  bit  of  w  is  1,  then  the  j«th  oracle  query 
in  the  computation  answers  “yes” , 

-  the  behaviour  of  the  machine  is  consistent  with  the  alleged  tape  contents 
and  alleged  oracle  answers  in  the  computation,  and 

-  all  the  “yes”  oracle  answers  (at  most  ||/2jfc||)  in  the  computation  are  correct. 

The  formula  ij){x,w)is  (note  i  >  2),  consisting  of  a  bounded  existential 

quantifier  around  some  sharply-bounded  universal  quantifiers  around  a  ^ 
oracle.  Furthermore,  ^  is  monotone  in  each  bit  of  w.  Since  w  has  only  ||£jfc  ||  many 
bits,  we  can  find  the  lexicographically  maximum  w  satisfying  ip  by  deterministic 
binary  search.  Having  found  this  w,  for  each  output  bit  r  in  parallel  we  query 
the  X'f.i  jb  oracle  'tp'{x,w,r),  which  asserts  that  there  is  a  computation  in  which 

“  the  computation  starts  with  x  on  the  input, 

—  for  all  jf  the  j-th  most  significant  bit  of  w  is  exactly  the  answer  to  the  j-th 
oracle  query  in  the  computation, 

-  the  behaviour  of  the  machine  is  consistent  with  its  alleged  tape  contents  and 
the  alleged  oracle  answers, 

—  all  the  “yes”  oracle  answers  in  the  computation  are  correct,  and 

—  there  exists  a  witness  to  the  final  query  that  leads  the  machine  to  output  a 
value  with  bit  r  set. 

By  maximality  of  lu,  there  is  a  unique  and  correct  such  computation  until 
the  final  witnessing  query.  Since  we  assumed  /  was  single- valued,  all  the  output 
bits  will  be  consistent,  even  if  the  circuits  to  compute  different  bits  happen  to 
find  different  witnesses  to  the  final  oracle  query.  Thus  /  6  ^ 

Krajicek  [16]  shows  that  the  predicates  in  7^‘^^-i[0(log)]  (i.e.,  V  with  O(log) 
queries  to  a  oracle)  are  precisely  those  Z'|'-definable  in  5^”^ .  His  technique 
actually  uses  ^FV  *-^[witfO{\og)],  but  the  restriction  to  predicates  rather  than 
functions  allows  him  to  eliminate  the  witnessing  and  multivaluedness  at  the  last 
moment.  Interestingly  enough,  the  above  proof  adapts  to  A;  =  2: 

Theorem 32.  Fori  >  2,  the  restriction  [luzt, O(log)]  to  single-valued 

functions  is  Df  2- 

Proof.  The  proof  of  Theorem  31  relies  on  A;  >  3  in  several  ways,  but  these  can 
all  be  fixed. 

(D):  The  previous  proof  takes  ||£jfc||  time  to  do  a  binary  search  for  each  of  the 
\\Ck\\  rows  of  the  array.  Since  ||£2||  is  not  closed  under  multiplication,  we  can 
no  longer  afford  this.  However,  by  our  convention,  for  some  term  t  €  £2,  each 
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oracle  gate  has  “depth”  ||t||  =  ^(log(n)).  But  a  0^2  circuit  has  depth  0(log(n)), 
so  there  must  be  a  constant  bound  d  on  the  number  of  oracle  gates  along  any  one 
path.  Arrange  the  array  in  d  meta-layers,  each  containing  one  row  of  oracle  gates 
and  O(log)  rows  of  ordinary  gates.  We  define  a  Aq  2  relation  cr'(n,r,c,b)  that 
does  the  same  thing  as  a  for  rows  containing  only  ordinary  gates.  We  then  define 
di  1^12  relation  (^'(n,r,b)  by  (3\w\  =  width  of  row  r)(Vc  <  \w\){Bit{c,w)  ^ 
cr'dxj,  r,  c,  b))).  The  algorithm  is  then  as  before,  but  for  ordinary  rows  we  simply 
make  a  witnessing  query  to  0'  rather  than  doing  a  binary  search  on 

(C):  We  can  still  assume  an  [wit,  O(log)]  machine  demands  witnesses 

only  on  its  final  query,  by  a  lemma  in  [16].  The  rest  of  the  proof  is  unaltered.  □ 

Corollary  33.  For  i  >2,  k  >2,  the  functions  E\^^-definahle  in  are 
Corollary  34.  For  i  >  2,  k>  2,  if  5^"^  =  then  ^ 

5  Links  to  Bounded  Arithmetic 

We  next  state  and  prove  a  theorem  analogous  to  that  in  [6],  demonstrating  a  close 
link  between  computation  in  this  parallel  hierarchy  and  the  bounded-arithmetic 
theories  i?j..  As  a  result,  we  can  simplify  and  strengthen  the  conservation  results 
of  [9]. 

Theorem35.  Fori  >  l,fc  >  3,  09  contains  exactly  the  /.-definable  fane- 
tions  of  R).. 

Fori  >  I,  contains  exactly  the  E\2-definahle  functions  of 

This  is  proven  by  methods  similar  to  those  in  [6,  1].  First,  DJ  (respectively 

is  equal  to  a  certain  recursively-defined  function  algebra.  Second, 
the  base  functions  of  this  algebra  are  provably  total  in  and  the  operators 
preserve  provable  totality,  so  any  function  in  the  algebra  is  provably  total  in  i?J.. 
Third,  any  U 11^  sequent  provable  in  J?j.  can  be  witnessed  by  a  function  in 
this  algebra. 

5.1  A  Function  Algebra 

We  define  a  hierarchy  of  function  algebras  {Ai^k}i>o,k>2  by  applying  two  simple 
operations  to  a  fixed  collection  of  base  functions  on  the  universe  of  finite  bit- 
strings. 

Definition  36.  The  BASE  functions  are  the  following: 

-  0,  1,  A,  the  constant  functions  (A  represents  the  empty  string), 

-  Lsp{x,y),  the  “least  significant  part”  function,  defined  to  be  the  rightmost 
I2/I  bits  of  X,  padded  on  the  left  with  zeroes  if  \y\  >  lx]. 

-  Msp{x,y),  the  “most  significant  part”  function,  defined  to  be  all  but  the 
rightmost  \y\  bits  of  x,  or  the  empty  string  if  \y\  >  |x|. 
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(X  iiw  =  X 
y  if  Lsp{w,  1)  =  0 
if  Lsp(w,  1)  =  1 

-  Conc{x,y)j  the  concatenation  of  x  and  y, 

-  Bh{x),  the  “back  half”  function,  defined  to  be  the  rightmost  bits  of  x, 

-  Fh{x),  the  “front  half”  function,  defined  to  be  the  leftmost  bits  of  x, 
-  Not{x)j  the  one’s  complement  of  a;, 

-  Or{x,y)^  the  bitwise  OR  of  x  and  y  (undefined  if  |a;|  jyl), 

-  And(a;,2/),  the  bitwise  AND  of  x  and  y  (undefined  if  |a;|  ^  |2/|), 

-  Iiiso{x),  a  string  exactly  twice  as  long  as  x  such  that  for  aJU  <  |a;|,  we  have 
Bit{2i,IiiSo{x))  -  0  and  Bit{2i  +  l,lQSo(a^))  =  Bit{i,x), 

-  Jnsi(a:),  defined  analogously  with  Bit{2i^Insi{x))  =  1. 

Definition 37.  A  function  /  is  defined  by  \Ck\-^ounded  divide-and-conquer  re¬ 
cursion  from  functions  g  and  h  if  there  is  a  term  t  e  Ck  such  that 


f{z,b,x)  = 


if  1^1  <  1^1. 


.  \  h{z,  6,  X,  f{Fbiz),  6,  x),  /(Bh(0),  6,  x))  otherwise, 

and  |/(2;,6,x)|  <  |<(2:,6,  x)|  for  all  2r,6,x. 

(The  somewhat  inelegant  |t|  bound  can  be  eliminated  by  the  use  of  “tiered 
recursion”,  as  in  [2,  4,  19].  For  the  purposes  of  this  paper,  the  arbitrary  bound 
is  less  onerous  than  the  additional  notation  and  explanation  needed  for  tiered 
descriptions  of  these  function  classes.) 

Definition  38.  The  function  algebra  Ai^k  is  the  closure  of  BASE  and  the  char¬ 
acteristic  functions  of  ^  relations^  under  the  operations  of  composition  and 
|£jfc|-bounded  divide-and-conquer  recursion. 

For  example,  we  define  a  function  CountUp{s)  e  Ao,2  which  rounds  |s|  up 
to  the  next  power  of  2,  and  produces  the  concatenation  of  the  binary  num¬ 
bers  0, 1, . . . ,  |s|  -  1,  each  embedded  in  a  block  of  ||s||  bits.  (This  example  not 
only  shows  some  of  DCR’s  power,  but  proves  useful  later  on.)  Let  SHL{x,  y)  = 
Lsp{Conc{x^y)^x)\  this  appends  y  to  x,  discarding  high  bits  so  the  result  still 
has  length  \x\.  Let  Zeroes(x)  =  Lsp(0,x),  a  string  of  zeroes  of  length  \x\.  Let 

f  A  if  |2;|  =  0 

Msk{z,b)=  <  SHL{Zeroes{b),l)  if  jzj  =  1 

[  Conc{SHL{Msk{Bh{z),b),0),SHLiMsk{Bh(z),b),0))  if  |z|  >  1, 

r  A  if  |2:|  =  0 

Aux(z,  6)  =  Zeroes{b)  if  jzj  =  1 

[Conc{AuxiBh{z),b),  Or{Msk{Bh{z),b},Aux{Bh{z),b)))  if  |2;|  >  1, 

and  finally  CountUp(z)  =  Aux(z,  |2:|).  I  leave  seeing  how  this  works  as  an  exercise 
for  the  reader. 


^  where  is  by  convention  {} 
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Lemma  39.  For  i  >  0  and  k  >  3, 

FoTi>0  and  k  =  2,  D?  ^  C  Ai,k  C  . 

Proof.  First  we  consider  the  z  =  0  case. 

(D):  Suppose  /  e  Aq,*.  All  the  BASE  functions  are  clearly  in  By 
Lemma  17,  □§  ^  is  closed  under  composition,  and  it  is  not  hard  to  see  that  it’s 
closed  under  |£fc|-bounded  divide-and-conquer  recursion  too.  (This  requires  k  > 
3  so  circuit  depth  is  closed  under  multiplication.  For  the  k  =  2  case,  since  PAfC 
is  closed  under  polynomial-bounded  DCR  [1],  we  can  conclude  Ao,2  Q  TMC.) 

(C):  Suppose  /  ^  Dq  fc  consider  its  Aq  i,  bit-graph.  Recall  that  Aq  j^  = 
ATIME(||£/b||).  At  the  cost  of  a  constant  factor  in  time,  we  can  assume  the 
ATM  is  strictly  alternating  between  existential  and  universal  states  and  that  it 
accesses  its  input  tape  only  at  leaves.  In  [4],  the  author  showed  how  to  simulate 
the  computation  of  such  an  ATM  within  a  function  algebra  similar  to  Aa^k-  Iii 
brief:  construct  a  binary  tree  data  structure  called  PATHS,  each  of  whose  leaves 
contains  an  encoding  of  the  path  from  the  root  to  that  leaf;  apply  £fc-bounded 
divide-and-conquer  recursion  to  this  encoded  path  to  determine  the  ATM  con¬ 
figuration  at  the  corresponding  leaf  of  the  computation  tree;  and  combine  the 
Boolean  results  using  a  constant-bounded  divide-and-conquer  recursion.  Thus 
^0  fc  —  Now  if  the  function  value  is  \s\  bits  long,  it  can  be  computed  from 
the  bit-graph  by  ls|-bounded  divide-and-conquer  recursion  on  CountUp(s),  com¬ 
puting  the  z-th  bit  at  leaf  i  and  concatenating  the  results  together,  again  as  in  [4]. 
Thus  C  Ao,fc. 

For  z  >  0,  the  3  direction,  recall  that  the  characteristic  functions  of 
relations  are  in  (and  by  definition.  The  proof  that  Ai^k  ^  ^i^k 

(for  fc  =  2,  is  almost  exactly  as  before. 

The  C  direction  is  a  little  more  complex.  Recall  from  Lemmas  13  and  20  that 
every  Df  function’s  bit-graph  is  in  ATIME{\\Ck\\)  with  oracle  states  for 
relations.  At  the  cost  of  a  constant  factor  in  time,  we  assume  the  levels  of  the 
ATM  rotate  strictly  among  universal,  existential,  parity,  and  oracle  states.  Let 
t  denote  the  bounding  term  for  all  the  oracle  states  in  the  machine,  rounded  up 
so  that  |t|  is  a  power  of  2.  We  construct  a  PATHS  tree  of  depth  equal  to  the 
time  bound  of  the  ATM  (counting  oracles  as  taking  ||t{|  time,  as  usual). 

Finally,  we  define  an  Ai^k  function  EVALTREE(T,  b,  x)  by  divide  and  conquer 
recursion.  (We  shall  eventually  invoke  it  with  T  equal  to  the  PATHS  tree,  b  any 
string  the  length  of  one  leaf  of  the  PATHS  tree,  and  x  equal  to  the  original 
input.)  As  defined  in  [4],  EVALTREE  returned  a  bit  indicating  whether  a  certain 
subtree  of  the  ATM’s  computation  tree  accepted  or  rejected.  In  this  setting,  it 
will  ultimately  do  the  same,  but  it  may  have  intermediate  values  up  to  \t{x)\ 
bits  long.  The  base  case  of  the  recursion,  exactly  as  in  [4],  treats  T  as  encoding 
the  path  to  a  particular  leaf  of  the  ATM’s  computation  tree;  it  follows  this 
path,  applying  the  left  or  right  next-step  function  at  each  step,  to  construct  the 
configuration  of  that  leaf,  and  finally  decides  whether  that  leaf  accepts  or  rejects. 

Since  \T\  is  cut  in  half  at  each  iteration,  ||T||  indicates  the  depth  of  a  tree,  up 
to  an  additive  constant.  Of  every  |lt||  +  3  levels  of  the  PATHS  tree,  ||t||  are  used 
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to  accumulate  the  answers  from  the  \t\  children  of  a  given  oracle  configuration, 
one  computes  parity,  one  OR,  and  one  AND.  The  final  value  of  EVALTREE  is 
1  or  0,  indicating  whether  the  ATM  accepts  or  rejects.  The  value  of  a  multi-bit 
function  can  be  pieced  together  just  as  in  the  i  =  0  case. 

□ 

This  concludes  the  proof  of  lemma  39. 

Lemma  40.  For  i  >  1,  Ai^2  = 

Proof.  Lemma  39  has  already  given  us  the  C  direction.  For  the  D  direction, 
which  resembles  the  proof  of  Allen’s  Theorem  1.3.3  [1],  let  f  G  be 

computed  by  a  (wolog,  levelled)  circuit  family  of  depth  0(log^‘(n)),  We  reason 
by  induction  on  j. 

If  j  =  1,  then  /  G  DJa?  and  therefore  /  G  Ai,2  by  Lemma  39.  If  j  >  1,  think 
of  an  /  circuit  as  divided  into  log(n)  meta-layers  of  depth  at  most  log^'“^(7i) 
each.  Let  ^  be  a  function  which,  given  the  values  of  all  the  inputs  to  a  meta¬ 
layer  and  a  number  indicating  which  meta-layer  it  is,  computes  all  the  outputs 
from  that  meta-layer.  This  function  can  be  computed  in  depth  0(log^"^(7i)), 
even  allowing  for  constructing  a  copy  of  the  relevant  part  of  the  /  circuit,  so  by 
the  inductive  hypothesis  g  G  The  desired  function  /  can  then  be  computed 
by  iterating  g  0(log(7i))  times,  which  we  can  do  by  DCR.  Since  the  f  circuit  is 
only  polynomial  size,  each  value  of  /  and  g  need  only  be  polynomially  many  bits 
long,  so  the  DCR  is  polynomially  bounded.  □ 

5.2  Definability  in 

Theorem 41.  For  i>l,k>2,  every  A*,*  function  is  E\y,-definable  in  Rj.. 

All  the  BASE  functions,  and  the  characteristic  functions  of  f,  relations, 
are  provably  total  in  foi  i>l,k>  2.  The  class  of  functions  -definable 
in  Rl  is  trivially  closed  under  composition,  so  we  need  only  to  prove  it  closed 
under  [-bounded  divide-and-conquer  recursion. 

Lemma 42.  For  i  ^  1,  k  >  2,  if  g  and  h  are  Ef  j^-definahle  in  RJ./  and  f  is 
defined  by  \Ck\-bounded  divide-and-conquer  recursion  on  them,  then  f  is 
definable  in  R\. 

Proof.  Suppose  g  and  h  are  defined  in  R*  by  the  r?  formulae  (^^(2r,x,3/)  and 
<t>h{z,  6,  X,  Wi,  U2,  y)  respectively,  and  /  is  defined  from  them  with  a  length  bound 
of  |s|.  We  define  a  formula  <l>f{z,  6,  x,  y)  that  T^^-defines  /  in  R|  by  asserting  the 
existence  of  an  encoded  binary  tree  T  representing  the  computation.  At  vertex 
i  of  this  binary  tree  we’ll  store  data  of  the  form  (yuZi)  (of  length  0(|s|  -b  |^|)) 
to  assert  that  f{zi,  6,x)  =  yi.  The  root  block  must  be  equal  to  {y,  z),  each  block 
{yi,Zi)  with  \zi\  >  |6|  must  be  appropriately  related  to  its  children  by  (fh,  and 
each  block  (yi,Zi)  with  \z\  <  |6|  must  satisfy  <j>g{zi,x,yi). 
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In  order  to  ensure  that  i?j.  can  prove  the  existence  of  such  a  T,  we  represent 
T  in  a  somewhat  inefficient,  “inflated”  way:  to  represent  a  tree  of  depth  ||2:||  with 
0{\s\  +  \z\)  bits  of  information  at  each  vertex,  we  use  an  array  of  blocks, 

each  of  length  0(l5|  + 12;|).  Each  subtree  is  represented  by  a  string  whose  first  half 
contains  the  data  for  the  subtree’s  root,  padded  with  garbage  bits  of  unspecified 
value.  The  third  quarter  of  the  string  represents  the  left  subtree,  and  the  fourth 
quarter  the  right  subtree.  This  enables  us  to  apply  divide- and- conquer  induction 
easily  to  the  whole  tree;  for  details  of  the  techniques,  see  [4].  Note  that  even  in 
inflated  form,  the  length  of  T  is  a  polynomial  in  \z\  and  l^j,  so  this  construction 
works  for  >  2.  □ 

Theorem  41  follows  immediately. 

5.3  Witnessing  R),  Proofs 

It  remains  only  to  show  that  every  function  i7|’  j^.-definable  in  Rl  is  in  Ai^k-  This  is 
done  by  a  witnessing  theorem  similar  to  those  of  [6,  1].  I  assume  familiarity  with 
their  techniques,  particularly  the  Witness  predicate.  To  extend  those  techniques 
to  the  present  setting,  we  need 

Lemma  43.  Fori>l,k>  2,  if  A{a.)  G  then  Witness^^  is  in 

Buss  proved  that  it  was  a  predicate,  but  we  have  no  evidence  that 
A^j^  =  A^f,.  Allen  stated  and  proved  the  lemma  for  i  =  l,fc  =  2,  but  there  is 
no’essential  reason  for  this  restriction.  The  proof  is  a  straightforward  induction 
on  the  syntax  of  A,  relying  on  ^’s  ability  to  do  simple  sequence  coding  and 
decoding.  It  uses  nothing  more  sophisticated  than  Lemmas  18  and  21. 

Theorem  44.  For  i  >  1,  k  >  2,  if  h  r(a)  — ►  A(a.),  where  F  and  A  are 
lists  of  (0  or  more)  formulae  with  free  variables  among  a,  then  there  is  an 
Ai^k  function  f  such  that 

Rl  h  Witness^^^{w,a)  D  Witness]^ ^{f{w, a), a)  . 

Proof  All  the  essential  ideas  appear  in  either  [6]  or  [1].  Buss  shows  how  to  wit¬ 
ness  axioms  and  formulae  derived  by  cosmetic.  Boolean,  and  quantifier  inferences. 
All  of  these  can  still  be  handled  within  which  we  know  is  contained  in  Ao,fe 
for  ifc  >  2.  Allen  shows  how  to  use  divide-and- conquer  recursion  to  provably 
witness  a  formula  in  whose  proof  the  last  step  is  i7i-DCI;  the  proof  extends  to 
i:^DCI  by  Lemma  43.  □ 

This  concludes  the  proof  of  Theorem  35. 

Since  X'-'j^-defines  precisely  the  functions,  we  immediately  get 

Corollary  45.  For  i  >  1,  k  >  3,  if  Sl  is  ^SEh^-conservative  over  Rl,  then 

^Ik  = 

Corollary  46.  For  i  >  1,  k  =  2, if  Sl  is  \f3E^i^-conservative  over  Rl,  then 
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5.4  Conservation,  Pro  and  Con 

The  characterization  of  Theorem  35  leads  to  a  new  proof  of  Theorem  21  of  [9]: 

Theorem  47  (Buss,  Krajicek,  Takeuti).  For  i>2,  k>S,  Rl  is  conserva¬ 
tive  over  with  respect  to  sentences. 

Since  the  consequences  of  Rl  correspond  exactly  to  the  ^  functions, 

it  suffices  to  show  that  can  T^^^^-define  the  functions  too.  We  stated 
this  before,  in  Corollary  33,  but  that  depended  on  the  result  we’re  now  trying 
to  prove.  Here’s  a  direct  proof  (which  incidentally  works  for  ib  =  2  as  well). 

Lemma 48.  Fori  >2,k>2,  5^"^  can  define  all  the  functions. 

Proof.  Let  /  €  Then  f  is  computed  by  some  -uniform  circuit  family  of 
genus  k  circuits  with  a  fixed  oracle  X.  The  existence  of  a  computation  to 

construct  and  simulate  this  circuit,  with  correct  oracle  queries,  is  certainly 
but  we  must  show  that  Sl~^  can  prove  that  existence.  I’ll  describe  the  algorithm 
first,  then  formalize  it  in  5^^  First,  assume  the  circuits  are  treelike  (which  is 
OK  for  and  levelled.  We’ll  scan  from  left  to  right  across  the  leaf  layer, 
at  each  step  computing  the  values  of  all  the  gates  in  the  whole  circuit  whose 
descendant  leaves  are  among  the  ones  we’ve  visited.  For  each  leaf,  this  requires 
adding  on  at  most  the  unique  path  from  that  leaf  to  the  root,  which  is  length 
||£a;||,  and  we  can  find  the  lexicographically  maximum  (and  hence  correct)  string 
of  this  length. 

Let  Aiic{r,  x)  be  the  number  of  gates  in  the  circuit  for  rc,  all  of  whose  descen¬ 
dant  leaves  are  numbered  r  or  lower.  Define  InfoTuple{w,  r,  x)  to  be  the 
formula  asserting  that 

-  u;  is  an  r-tuple  of  bit  strings  of  lengths  Aizc(l,  x),  Anc{2,  x)  ~  Anc{l,x), . . . , 
Anc{r,  x)  -  Anc{r  -  1,  x)  respectively,  and 

-  for  all  j  <  |w,.|,  if  the  j-th  most  significant  bit  of  Wr  is  1,  then  the  j-th 
ancestor  gate  of  leaf  r  outputs  1  when  given  input  as  specified  in  w. 

Thus  InfoTuple  asserts  that  u;  is  a  possible  assignment  of  values  to  the  ancestors 
of  leaves  l,...,r,  in  which  at  least  all  the  “yes”  answers  are  correct  for  their 
presumed  inputs. 

Define  GtrTuple(WiW')  to  be  a  A\  formula  asserting  that  for  some  r,  r', 

-  w  and  w’  are  tuples  of  r  and  P  elements  respectively, 

-  r  >r‘, 

-  (Vj  <  P)(wj  >Wj)j  and 

-  (r  >  P)  V  (3j  <  P)(wj  >  Wj). 

That  is,  each  bit-string  entry  of  tn,  treated  as  a  binary  number,  is  at  least  as 
great  as  the  corresponding  entry  of  w',  and  either  at  least  one  entry  of  w  is 
greater  than  the  corresponding  entry  of  or  w  is  longer  than 
Now  define  InfoSeq(Wj  r,  x)  to  be  the  formula  asserting  that 
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-  VT  is  a  sequence  of  exactly  r  tuples, 

“  (Vj  <  r)InfoTaple{Wj,j^x),  and 

-  (Vj  <r  -  l)GtrTaple{Wj^ijWj). 

By  LIND  on  {3W,  T)InfoSeq{W,  r,  x),  Sl  can  prove  there  is  a  maximum-length 
W  satisfying  InfoSeq{W^r^  x)  with  r  bounded  by  the  number  of  leaves  in  the 
circuit;  note  that  each  tuple  has  |£jfc|  entries,  each  entry  is  bounded  in  numeric 
value  by  l£fe|,  and  each  tuple  in  such  a  sequence  must  have  a  distinct  set  of 
entries.  Let  w  denote  the  last  element  of  this  maximum-length  W.  It  must  contain 
a  bit-string  for  each  leaf,  or  w  (and  therefore  W)  could  be  extended  by  adding 
on  a  string  of  zeroes  for  the  next  leaf.  The  first  bit-string,  Wi^  must  have  the 
maximum  possible  value  consistent  with  input  x^  or  W  could  be  extended  by 
increasing  wi.  Thus  all  the  “no”  answers  in  wi  must  be  correct.  Similarly,  W2 
must  have  the  maximum  possible  value  consistent  with  x  and  wij  and  so  on; 
after  \Ck\  steps  we  conclude  that  every  gate  value  in  w  is  correct.  Extracting  the 
value  of  the  function  is  then  straightforward.  O 

Together  with  the  first  part  of  Theorem  35,  this  implies  Theorem  47.  The 
second  part  of  Theorem  35,  together  with  Corollary  33,  gives  us  some  insight  as 
to  why  it  h£LS  been  so  difficult  to  strengthen  Theorem  47  to  fc  =  2: 

Theorem 49.  For  i  >  2,  if  is  conservative  over  Sl~^  with  respect  to  V3X'J’2 
sentences,  then  =  0^2;  o,nd  hence  =  Ai  2- 

Since  ^^2  —  would  imply  a  relativized  collapse  of  A/’C 

down  to  Of  course,  Wilson  showed  such  a  relativized  collapse  in  [26],  but 
he  constructed  only  a  recursive  oracle;  it  would  be  surprising  to  find  such  an 
oracle  within  the  polynomial  hierarchy.  Conversely,  if  an  oracle  collapsing  FMC 
to  were  found  within,  say,  X'f ,  it  would  imply  conservativity  between 

and  for  all  j  >  i. 


6  Links  to  Complexity  Hierarchies 

The  authors  of  [17,  18]  defined  a  “student-teacher  game” ,  a  dialogue  between 
an  omniscient  teacher  and  a  computationally  limited  student  assigned  to  find 
the  best  solution  to  some  problem.  Each  time  the  student  presents  a  subopt imal 
solution,  the  teacher  replies  with  “no,  that’s  not  optimal;  see,  here’s  a  better 
solution,”  and  the  student  may  then  use  this  better  solution  in  constructing 
the  next  solution.  (In  fact,  the  student  may  simply  parrot  it  back  as  the  next 
solution.  This  algorithm  is  called  the  trivial  student.)  With  the  aid  of  theorem  35 
and  this  computational  model,  we  can  prove  several  results  analogous  to  those 
of  [18]. 
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6.1  A  Student-Teacher  Witnessing  Theorem  for 

Theorem  50.  For  i>l,k>3,  and  <!>  €  377?;^,  if  Si  I-  {3y)('iz)<l>{a,y,z),  then 
there  are  functions  fi,...fj  such  that’ 


SI  h  0(o, /i (a, 6i))  V  fiia,  6i), 62)  V  ...  V  f>{a,  fj{a,  61 , . . .  bj-i),  bj)  . 

That  is,  there  is  a  student  that  finds  a  witness  for  {3y){'dz)4i{a,y,  z)  within 

some  constant  number  r  of  rounds,  provably  in  . 

The  proof,  which  appears  in  detail  in  the  journal  version,  resembles  that  of 
Theorem  A  of  [18],  with  two  main  differences.  First,  in  place  of  the  equational 
theory  PVi+i,  which  contains  a  function  symbol  for  every  function  definition 
in  a  Cobham-style  algebra  for  ,  we  define  an  equational  theory 
containing  a  function  symbol  for  every  function  definition  in  Second,  we 

replace  prefix  induction  by  divide- and-conquer  induction,  and  successor  induc¬ 
tion  by  prefix  induction,  throughout. 


6.2  Collapsing  Bounded  Arithmetic 

At  this  point,  a  previous  version  of  this  paper  contained  an  adaptation  of  the  ar¬ 
gument  of  [18]  that  T2  =  82'^^  ^  C  Aj^j/poly.  However,  Buss  [8]  recently 
gave  a  simpler  proof  of  a  stronger  result,  so  I’ll  adapt  that  technique  instead. 
The  main  substantive  difference  between  the  following  proof  and  Buss’s  is  the 
replacement  of  prefix  induction  with  divide  and  conquer  induction;  however,  the 
proof  technique  is  interesting  enough  to  warrant  another  exposition. 

Theorem  51.  Fori>l,k>  3,  =  then 

1.  SI  proves  that  every  formula  B{x)  is  equivalent  to  a  formula  of  the  form 

C{x,A{x))  where  A  is  a  of  depending  only 

on  |a:|,  and  C  €  Hi+i/ 

2.  Si  proves  C  A^i_^,^  Jpoly; 

4-  SI  =  Sk,  which  is  therefore  finitely  axiomatized;  and 
5.  for  every  bounded  formula  in  Ck ,  Si  proves  it  equivalent  to  a  Boolean  com¬ 
bination  of  B^^2,k  formulds. 

The  idea,  as  in  other  proofs  (e.g.  [18,  8])  involving  student-teacher  games, 
is  to  play  the  omniscient  teacher,  giving  away  as  little  information  as  possible, 
and  playing  for  long  enough  that  the  student  must  show  some  semblance  of 
originality.  Later,  a  won-omniscient  teacher  faced  with  a  new  problem  can  enlist 
the  aid  of  this  same  demonstrably  nontrivial  student  and  solve  the  problem.  But 
before  we  can  apply  this  technique,  we  must  define  some  preliminary  notions. 
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Definition  52.  Ilf  is  the  class  of  quantified  Boolean  formulae  in  prenex  form 
containing  i  blocks  of  like  quantifiers,  starting  with  a  V. 

TRU^{<l>j  w)  is  a  formula  stating  that  0  encodes  a  Tlf  formula  and  w  encodes 
a  satisfying  assignment  of  its  free  variables. 

SAT^{4>)  is  the  formula  {3w  <  (j))TRU^{ij),w). 

Note  that  TRW  and  SAT^  are  formulae  in  the  language  of  bounded  arith¬ 
metic,  while  (j>  is  a.  numeric  encoding  of  a  quantified  Boolean  formula  (but  we 
shall  often  identify  it  with  that  quantified  Boolean  formula,  for  convenience). 
Note  also  that  we  assume  the  encodings  of  (j>  and  w  are  reasonably  efficient, 
encodable  and  decodable  in  ALOGTIME,  and  that  any  assignment  of  the  free 
variables  in  (j>  has  an  encoding  numerically  less  than  0  itself.  We  assume  standard 
functions  for  encoding  and  decoding  sequences,  e.g.  SeqLen{s)  =  the  length  of 
the  sequence  encoded  by  s. 

It  is  well  known  that  SAT^  is  complete  for  2’  therefore  for  X'i+1,2 
formulae,  under  polynomial-time,  many-one  reductions.  The  techniques  of  the 
proof  are  not  computationally  demanding,  and  can  be  formalized  in  ^2, 
or  even  (with  care)  i^,  and  the  reductions  can  be  weakened  to  AfC  or  even 
ALOGTIME.  Indeed,  for  A:  >  2,  5AT*  is  complete  for  and 

many-one  reductions;  the  larger  growth  rates  are  needed  only  during  the 
reduction.  And  by  standard  techniques,  TRW  is  with  respect  to  R^. 

The  problem  we  shall  set  our  student  is,  given  a  sequence  of  n  encoded 
formulae  ,  02, . . . , to  find  the  longest  initial  sequence  0i , 02,  •  •  • ,  0m  all  of 
which  are  satisfiable,  and  produce  witnesses  iui,'u;2,  ■  •  • ,  for  them.  (To  avoid 
funny  behaviour  at  0,  we  give  the  student  a  witness  wi  for  0i  for  free.)  However, 
we’ll  accept  a  witnessed  initial  sequence  if  it  is  within  a  factor  of  2  in  length  of 
the  longest  one.  That  is,  the  student  is  to  realize  the  principle: 

*(V0i,...,0„  <  2^^^){3wi, . . ,  ^Wm  <  0>)Max5oin(a,  <0,w) 
where  MaxSoln{aj  <0,  w)  is  defined  by 
SeqLen{w)  <  SeqLen{(j))A 

(Vj  <  SeqLen(<0))0j  <  2l‘*l  A  (Vj  <  SeqLen{w))wj  <  (j>jh 

(Vj  <  SeqLen{w))TRW{(l)jyWj)A 

(SeqLen(0)  >  2  •  SeqLen{w)  D  <  2  •  SeqLen{w))->SAT^{<t>j). 

Note  that  MaxSoln  e  so  (*)  is  a  statement,  and  provable  in 

R^f'^  by  maximizing  p<  ||0||  in  the  formula 

(3w  <  <0)(|SeqLen(w)|  =  p  A  (Vj  <  SeqLen{w))TRW{(l)j,Wj))  . 

If,  as  we  shall  assume  for  the  rest  of  this  theorem,  SI  =  R]f^  and  therefore 
SI  could  also  prove  principle  (*),  then  Theorem  50  would  imply  the  existence 
of  a  ^  student  who  witnesses  it  in  some  constant  number  r  of  rounds.  If  in  a 
given  round,  the  student  proposes  a  sequence  of  witnesses  , . . . ,  Wm  which  is 
not  “nearly  maximal”,  the  teacher  is  obliged  to  demonstrate  that  by  providing 
witnesses  for  at  least  the  first  2m  formulae.  Thus  Si  =  Rl'^^  implies  there  are 
functions  /i, . . .  ,/r,  Ef^i-defined  in  such  that,  for  any  witnesses  w  for  (0, 
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MaxSoln(a,  0,  /i (a,  0,  wi))  V 
MaxSoln{a,  0,  /2(a,  0,  wi , . . . ,  iU2mi ))  V 
MaxSoln{a,  0,  /a  (a,  0,  «;i , . . . ,  t£;2m2 ))  V 

V  MaxSoln{a,  0,  /r(a,  0,  , . . . ,  t(;2m^_i )) 

where  =  SeqLen{fj  (a,  0,  tui , . . . ,  ti;2mj_i  ))• 

Define  an  original  witness  to  be  any  witness  the  student  provides  without 
first  being  given.  The  trivial  student,  the  one  who  never  provides  an  original 
witness,  will  produce  mi  =  1,  m2  =  2,  m3  =  4, ...,mr  =  2^“^.  So  to  coax 
an  original  witness  from  our  student,  it  suffices  to  let  n  >  2^  and  make  all  the 
formulae  satisfiable.  Define  the  formula  FindsOrigBy{a,m,  (0i, . . . ,  0s))  to  be 

(Vw  <  0  <  2l“l)((Vi  <  SeqLeii{<j)))TRW((l>j,Wj))  D 
{SeqLen{fi{a,  >  1  V 

SeqLen{f2{a,(j),  (wi,W2)))  >  2  V 
SeqLen(f3{a,(t>,(wi,W2,W3,W4))  >4  V... 

V  SeqLen{fm{a,  0,  ('lui, . . .  ,ii;2— 1))  >  2”^-^)) 


This  states  that,  no  matter  what  witnesses  are  provided  for  the  formulae  0, 
the  game  will  provide  at  least  one  original  witness  within  the  first  m  rounds,  and 
therefore  an  original  witness  for  one  of  the  first  2”^  formulae.  The  assumption 


SI  =  implies 


Si  h  (V01, . . . ,  02^  <  2l"l)  ( /\  SAT\<l>j))  D  FindsOrigByia, r,  0) 


j=i 


It  is  conceivable  (assuming  FindsOrigBy{a,m,  <j}))  that,  although  the  game 
is  guaranteed  to  give  an  original  witness  for  one  of  the  first  2”^  formulae,  the 
student  uses  the  later  formulae  to  find  it;  for  example,  suppose  the  last  formula 
in  the  sequence  encodes  a  polynomial- time  algorithm  for  SAT!  The  truth  value 
of  FiiidsOngBy(a,  m,  0)  may  therefore  depend  on  all  of  0,  even  though  only  the 
first  2^  0’s  are  actually  treated  as  formulae  to  be  witnessed.  We  therefore  distin¬ 
guish  the  parts  of  0  that  might  be  witnessed  from  the  parts  serving  only  as  “ad¬ 
vice”  to  the  computation,  and  write,  e.g,  FiDdsOrigBy(aym,  (0i, . . . , 02m),  A). 

Now  consider  the  formula  PreAdvice(a,  m.  A)  defined  by 


(2m 

{^SAT'{4>j))  D  FindsOrigBy(a,m,<p,A) 


This  states  that,  for  any  sequence  of  2^  satisfiable  formulae  0,  and  any  witnesses 
for  those  formulae,  the  game  with  advice  A  will  provide  an  original  witness  for 
one  of  0. 
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Recall  that 


(A  SAT\<j>j))  D  FmdsOrigBy{a,r,<j)) 


Rephrasing  this  in  terms  of  advice,  SI  h  PreAdvice{ajrj{)).  Since  r  is  a  con¬ 
stant,  SI  proves  without  induction  that  there  is  a  minimum  m  <  r  such  that 
(3i4)PreAdvice(a,m,  A).  Let  Advice{a,m^  A)  assert  that  m  is  such  a  minimum, 
and  A  is  a  corresponding  advice  string.  Then  SI  h  (3m,  A)Advice(a,m,  A). 
Finally,  given  a  formula  if)  e  Sf  and  “auxiliary”  formulae  ^i, . . .  ,<^2^-1? 
TestSeq(0,^)  be  the  sequence  of  length  2""  formed  by  concatenating  <p  with 
2”^“^  copies  of  'll). 


Now  we  are  ready  to  prove  the  theorem.  I  claim  that  S^  proves 


(Advice(a,  m,  A)  A  ^  <  2l®l)  D 
{SAT\i))  ^  (V(/.i, . . . ,  02—1  <  2l“l) 

(Aj=i  'S'AT*(0j)  D  FmdsOrigBy{a, m,  TestSeq(0, 0),  A). 


Proof.  Assume  Adyice(a,  m.  A)  and  ip  <  2l“l.  If  ip  is  satisfiable,  then  for  any 
satisfiable  sequence  (pi,..  .,^-1,  TestSeq{<p,ip)  is  a  sequence  of  2'^  satisfiable 
formulae  and  so,  by  PreAdvice(a,  m,  A),  an  original  witness  will  be  found  for  it. 

On  the  other  hand,  if  ip  is  not  satisfiable,  then  any  original  witness  for 
TestSeq{(p,ip)  must  actually  witness  one  of  the  first  2”^”^  formulae  <p,  using 
Ip  only  as  extra  advice.  Thus  FindsOngBy(a,m,  TestSeq(<p,ip),A)  is  equivalent 
to  FindsOngBy(a,m  -  l,<p,(A,ip}).  However,  by  minimality  of  m,  there  is  no 
advice  A!  for  which  PreAdvice{a,m  -  1,A')  holds.  In  particular,  for  the  ad¬ 
vice  A'  =  {A, Ip),  there  must  exist  a  satisfiable  sequence  (pi,...,  (p2m-i  such  that 
FindsOrigBy(a,  m  —  1, 0,  A')  is  false.  □ 


Thus  in  the  presence  of  the  “advice”  A,  which  depends  only  on  a,  the 
complete  formula  SAT^{ip)  is  equivalent  to  the  formula 


(2m-l 

SAT\(pj)  D  FindsOrigBy{a,Tn,TestSeq{<p,ip),A) 


This  proves  the  first  part  of  the  theorem:  if  B{x)  €  then  by  the  com¬ 
pleteness  of  SAT^,  there  is  a  poly-time  function  /  such  that  B{x)  ^  SAT^{f{x)). 
We  can  think  of  the  advice  A  as  a  (not  necessarily  single-valued)  function  of 

where  (7(0,  A)  is  the  above  formula. 

The  second  part  follows  immediately.  In  fact,  since  the  reduction  function  / 
above  can  be  made  quasilinear  [24],  we  get  -  -^^4.i^fc/quasilinear. 

The  third,  fourth,  and  fifth  parts  follow  much  as  in  [8], 
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7  Conclusions 

The  results  of  this  paper  may  be  viewed  as  filling  several  holes  in  our  under¬ 
standing  of  theories  of  bounded  arithmetic: 

—  easily-described  computational  characterizations  of  the  V3X'|  consequences 
oiRl 

—  the  provable  single- valuedness  of  witnessing  functions  for  these  consequences, 

-  easily-described  computational  characterizations  of  the  consequences 

oiSl 

~  simpler  complexity- theoretic  implications  of  than  previously  known, 

-  complexity-theoretic  implications  of  i?!  = 

-  complexity-theoretic  implications  if  the  known  VBZ'f^i-conservativity  be¬ 
tween  SI  and  were  extended  down  to  A;  =  2,  and 

—  complexity-theoretic  implications  of  =  R]^^ . 

There  remain,  of  course,  a  few  much  smaller  but  still  nagging  holes.  One  is  our 
inability  so  far  to  prove  analogues  of  Theorems  50  and  51  for  A:  =  2,  precisely 
the  case  of  the  greatest  computational  interest.  Another  remains  whether  R^^ 
is  indeed  V3i7f^.]^-conservative  over  S^- 

Krajicek  [16]  shows  that  the  equivalence  =  Tj,  like  the  equivalence  52"^^  = 
TJ,  would  have  startling  complexity- theoretic  implications:  any  problem  solvable 

yip 

in  V  could  be  solved  with  only  0 (log)  many  queries  to  its  oracle.  (It  is  not 
known  whether  this  would  imply  the  collapse  of  the  polynomial  hierarchy.)  The 
author  has  not  yet  found  a  natural  complexity  class  corresponding  to  the 
definable  functions  of  R^.  Such  a  class  would  provide  stronger  (and  presumably 
still  less  plausible)  implications  of  Ri  =  Si. 

The  larger  questions  not  addressed  in  this  paper  remain  stubbornly  unsolved: 
are  there  any  equivalences  among  the  theories  and  is  the  theory  Tk  = 

Sk  =  Rk  finitely  axiomatizable  for  any  A:,  and  do  the  corresponding  complexity 
classes  collapse? 
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Program  Extraction  from  Classical  Proofs 
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Abstract.  Different  methods  for  extracting  a  program  from  a  classical 
proof  are  investigated,  A  direct  method  based  on  normalization  and  the 
wellknown  negative  translation  combined  with  a  realizability  interpreta¬ 
tion  are  compared  and  shown  to  yield  equal  results.  Furthermore,  the 
translation  method  is  refined  in  order  to  obtain  optimized  programs.  An 
analysis  of  the  proof  translation  shows  that  in  many  cases  only  small 
parts  of  a  classical  proof  need  to  be  translated.  Proofs  extracted  from 
such  refined  translations  have  simpler  type  and  control  structure.  The 
effect  of  the  refinements  is  demonstrated  at  two  examples. 


1  Introduction 

It  is  well  known  that  from  a  classical  proof  of  Va;3?/B(a;,  y),  B  quantifier-free, 
one  can  extract  a  program  t  such  that  Va;  B{x,  tx)  holds.  We  discuss  two  possi¬ 
bilities  to  do  this.  1.  A  direct  method,  which  uses  the  clcissical  proof  and  proof 
normalization  directly  as  an  algorithm.  2.  A  translation  of  the  classical  proof 
into  an  intuitionistic  one  from  which  via  a  realizability  interpretation  a  program 
can  be  extracted.  We  show  that  both  methods  yield  the  same  algorithm. 

Furthermore  we  try  to  answer  the  question  wether  “programs  from  classical 
proofs”  is  a  useful  device  practically.  We  apply  the  proof  translation  to  a  simple 
but  informative  example:  We  prove  classically  that  w.r.t.  an  unbounded  function 
/:IN  — IN  such  that  /(O)  =  0  each  n  has  a  root  m,  i.e.,  /(m)  <  n  <  f(m  +  1) 
holds.  We  translate  the  proof  and  extract  a  program  root:  IN  — IN  (depending  on 
/)  such  that  /(root(n))  <  n  <  /(root(n)  + 1)  holds  for  all  n.  It’s  interesting  that 
the  classical  proof  is  extremely  easy  and  short  (even  if  fully  formalized);  consi¬ 
derably  shorter  than  the  intuitionistic  proof  one  would  give  intuitively.  However 
the  extracted  program  is  unnecessarily  complicated.  We  take  this  as  a  motivation 
to  study  refinements  of  the  proof  translation  yielding  simpler  programs. 

Program  extraction  can  be  messy  for  mainly  two  reasons:  1.  A  completely 
formalized  proof,  using  the  basic  axioms  of  arithmetic  only,  will  in  general  be 
extremely  long.  This  can  be  remidied  by  introducing  additional  global  assump¬ 
tions  which  are  of  such  a  form  that  they  do  not  spoil  the  extraction.  2.  When 
translating  a  classical  derivation  into  an  intuitionistic  one,  each  atomic  formula 
P  is  replaced  by  (P  — »■  A)  — ^  A,  where  A  is  the  existential  formula  we  want  to 
prove.  Thus  existential  formulas  are  spread  all  over  the  derivation  and  therefore 
each  sub  derivation  gets  computational  content.  This  means  that  the  extracted 
program  will  be  at  least  as  long  and  complicated  as  the  proof.  Furthermore  one 
has  to  pay  for  the  additional  assumptions  introduced  in  1.,  since  their  transla- 
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tions  have  to  be  proved.  In  general,  these  proofs  use  case  splittings  which  later 
show  up  in  the  program. 

In  this  paper  we  propose  a  refined  proof  translation  which  does  not  replace  all 
atoms  P  hy  {P  A)  A.  By  R  simple  syntactical  analysis  of  the  assumptions 
used  in  the  derivation  one  can  determine  a  set  of  critical  atoms  which  suffice 
to  be  replaced  in  order  to  make  the  translation  work.  Applying  this  refined 
translation  to  our  root  example  simplifies  the  resulting  program  drastically,  A 
second  example,  concerned  with  a  search  in  binary  trees,  shows  a  similar  effect. 

It  would  be  interesting  to  see  if  our  refined  method  can  be  applied  successfully 
to  larger  examples  too.  An  candidate  might  be  the  classical  proof  of  Higman’s 
Lemma  [3],  [7].  This  proof  has  been  translated  and  implemented  in  the  Nuprl 
system  by  [6] .  It  is  not  known  how  the  translated  proof  (which  is  extremely  big) 
is  related  to  the  known  constructive  proofs  of  Higman^s  Lemma  [8],  [1].  A  refined 
translation  might  help  answering  this. 

2  Preliminaries 

Our  basic  logical  calculus  is  the  A— ►V-fragment  of  minimal  natural  deduction 
for  first  order  logic  over  simply  typed  lambda-terms.  Classical  and  intuitionistic 
arithmetic  and  extensions  thereof  will  be  introduced  via  axioms. 

Types  are  the  ground  types  boole  and  nat,  p  x  a  and  p  a. 

Terms  are  the  terms  of  GddePs  system  T,  i.e.  built  from  typed  variables 
the  constants  true‘>°“'',  false*>“°'*,  0"^‘,  5"“*-*"^*,  (recur¬ 
sion),  (case  analysis)  by  pairing  (r,  s)'”“',  projection  ;r,(r'’''’‘'’‘)'’S 

abstraction  '(Xx^r^y~"^  and  application  (r^^^^s^y. 

We  have  the  usual  conversions  (writing  <  +  1  for  St) 

(Xxr)s  — ►  r[s/x], 

7r,(ro,ri) n, 
r, 

Rxi^t,prs{t  -f  1)  ->•  st{Rm.t,prst), 

Rhode, prstme  r, 

72boole,p^^  ^ 

It  is  well  known  that  each  term  reduces  to  a  unique  normal  form  w.r.t.  these 
conversions  (cf.  [II]).  To  simplify  the  calculus  we  will  identify  terms  with  the 
same  normal  form.  We  will  write  =  for  equality  modulo  normal  forms  and  =  for 
syntactical  identity. 

Atomic  formulas  are  1.  and  P(^  where  P  is  a  predicate  symbol  and  f  is  a  list 
of  terms.  The  types  pi  of  the  terms  U  are  specified  by  the  arity  (^  of  P.  We  have 
at  least  one  predicate  symbol  atom  of  arity  (boole).  The  formula  atom(i)  will 
play  the  same  role  as  usually  <  =  0  does.  Frequently  we  will  abreviate  atom(i) 
by  t.  We  assume  that  we  have  assigned  to  every  predicate  symbol  P  /  atom  of 
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arity  (p)  a  closed  term  tp  of  type  p  — »•  boole  defining  the  characteristic  function 
of  P.  This  means  that  we  consider  only  decidable  predicates. 

Formulas  are  built  from  atomic  formulas  by  conjunction  A  A  By  implica¬ 
tion  A  B  and  universal  quantification  A.  Negation,  disjunction  and  the 
existential  quantifier  are  defined  by 

-^A:=A-^  1, 

Aw  B  :=  -iA  A  -^B  —*■  -L, 

3x  A  :=  -Wx  ->A. 

By  WA  we  denote  the  universal  closure  of  the  formula  A.  A  11 -formula  is  a 
formula  of  the  form  Wx  C,  where  C  is  quantifier-free. 

Axioms  are  divided  into  two  groups. 

Induction  axioms: 

Indn,^:  V.A[0/n]  — >■  (Vn.A  A[n-{-  1/n])  Vn  A, 

Indp,^:  V.A[true/p]  — ^  A[false/p]  -^WpA. 

n -axioms:  All  these  axioms  are  closed  11 -formulas. 

T:  atom(true), 

-•F:  -'atom(false), 
and  for  each  predicate  symbol  P 

Efqp: 

Stabp!  P(x)  P(x), 

Decp^o*  Vx.P(^  — )■  atom(<px), 

Decp^i:  Vf.atom(tpic)  — )■  P(x). 

Clearly  there  is  a  lot  of  redundance  in  these  axioms.  However,  since  the  choice  of 
the  axioms  in  a  particular  proof  influences  the  extracted  program,  it  is  important 
to  have  a  rich  axiom  system. 

Derivations  are  built  from  assumptions  and  eixioms  by  the  introduc¬ 
tion  and  elimination  rules  for  A,  and  V: 

{d^ye^)^^^y 

{\xP  d^y^"  ^  y 

with  the  usual  variable  condition  in  the  case  of  (Aa?  We  will  write  P  f-  A 

if  there  is  a  derivation  d^  with  free  eissumptions  among  P.  It’s  easy  to  see  that 
we  caii  derive  stability  -i-iA  — »•  A  for  all  formulzis  A.  Hence  we  have  full  classical 
arithmetic.  Furthermore  each  quantifier-free  formula  A  is  decidable,  i.e.,  there 
is  a  boolean  term  such  that  h  A  <-+•  atom(t>i).  This  can  be  used  to  do  case 
splitting  according  to  quantifier-free  formulas  A,  i.e.  for  every  formula  B  we  can 
prove 

GasesA.B:  (A  — >■  B)  (-lA  B)  B. 
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The  derivation  Cases^^B  (which  we  will  use  later  on)  is  given  by 

Awi,  U2-Ind  ^(Au3Au4.W3T)(Aw5A«6.U6”’F)fA(Au7-'Wl(doU7))(Au8-‘W2(dlU8)) 

where  derivations  which  exist  according  to 

the  decidability  of  A,  and  the  axioms  and  assumption  variables  with  indices  are 
(writing  t  for  atom(i)) 

►B)— j 

..~iA-*B  ,.true-^B  ,.-itrue-+B  false-^B  ,,-lfalse-^«B  ,,<>4 
“1  j  “2  5  “3  )  “4  J  “5  )  “6  j  “7  ^8  * 

Conversion  for  derivations  is  defined  similary  as  for  terms  in  the  usual  way, 

7r,((do)  di))  — >  di, 

(Au  d[e/u], 

{Xxd)'^^^t~^d[t/xl 
Indn,^rdeO  d, 

lTLdn,Arde{t  +  1)  ^  et{liidn,Ardet) 

Indp^^rde  true  — >  d, 

In dp^^fde  false  e. 

Again  it  can  be  shown  by  standard  methods  — just  as  for  Godel’s  T  —  that  any 
derivation  term  in  arithmetic  has  a  unique  normal  form  w.r.t.  these  conversions. 

Iniuiiionistic  arithmetic  is  obtained  by  extending  the  calculus  above  by  a 
constructive  or  strong  existential  quantifier  written  3*  (as  opposed  to  3  defined 
by  -iV-i).  We  add  axioms  corresponding  to  the  usual  introduction  and  elimination 
rules  for  3* . 

3;;+:  Vix.A-^3*xA, 

X:a.b-  A)  -  ('ixO.A 

Of  course,  in  general,  stability  is  underivable  for  formulas  containing  3*.  Con¬ 
structive  disjunction  V*  can  be  defined  by 

Ay*  B  3*p.(p  =:  true  — ^  A)  A  (p  =  false  — >■  j9). 

Program  extraction  from  derivations  with  3*  is  done  via  a  modified  realizabil¬ 
ity  interpretation  due  to  Kreisel  (cf.  [4]).  First  we  associate  with  each  formula  A 
(possibly  containing  3*)  a  list  of  types  t{A)  as  follows. 

T{R{i))  ;=  e, 

where  e  denotes  the  empty  list,  and  if  r[A)  =  p  and  t[B)  =  (Ti,  . . . ,  we  let 

t{A-*B)  :=p-KTi,...,p-i-tr„, 
t{A  a  B)  :=  p, 

r(Vx^5)  :=p-^o-i,...,p^(7n, 
t{3*  x^  B)  :=  p,  d. 
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Instead  of  p  — >■  cri, . .  .,p  — or„  we  will  sometimes  write  p  a.  To  give  some 
examples,  let  n^m^k  be  of  type  nat.  Then 

r(Vn3*mi2(n,  m))  =  nat  — ^  nat, 

T(\/n3*m3*k  R(n,  m,  A?))  =  (nat  — >•  nat),  (nat  — »■  nat), 
T(in3*mR{n,  m)  -4  3*k  Q{k))  =  (nat  — »•  nat)  — f  nat. 

For  every  formula  A  (possibly  containing  3*)  and  every  list  f  of  type  t{A)  we 
define  a  formula  rmri4  without  3*,  to  be  read  f  modified  realizes  A, 

emrR{f)  :=  R(i)y 

ri , . . . ,  r„  mr  (A  ”4  5)  :=  'ix.x  mr  A  -4  ri£, . . . ,  VnX  mr  5, 
r,  smr(A  A  B)  :=  fmr  A  A  smr5, 
ri , . . . ,  Tn  mr  B  :=  'ix^  ri ar, r„ar  mr  B, 

r,  smT3*x^  B  :=  smTB[r/x]. 


Note  that  if  A  does  not  contain  3*  then  r(A)  =  e  and  emr  A  =  A  . 

Definition.  Assume  that  to  any  assumption  variable  we  have  assigned  a  list 
xl^^^  =  x^\,...,  x^i  of  distinct  variables,  where  /^i, . . . ,  =  t{B).  Relative  to 

this  assignment  we  define  for  any  derivation  its  extracted  terms  ets{d^),  by 
induction  on  d^.  If  r(A)  =  <ti,  . . . ,  cta?,  then  ets(d^)  will  be  a  list  . . . ,  . 


ets(Indn,yi) 
ets(Indp,A) 
ets(i7-axiom) 
ets(u'^) 
ets(Au'^  d^) 
ets(d^~*^ 
ets((d"^,  e^)) 
ets(7ro(d'^^^)) 
ets(7ri(d'^^^)) 
ets(A£^  d^) 
ets{d'^^" 
ets(3:+) 

MK7a,b) 


—  Aaji2nat,r(>l)  > 

=  Aa?Rboole,r(j4)) 

= 

—  J 

=  ets(c?), 

=  ets(d)ets(e), 

=  ets(c?^),ets(e^), 

~  the  head  of  ets(d^^-®)  of  same  length  as  r(A), 
=  the  tail  of  ets(d^^^)  of  same  length  as  r(J5), 
=  Aic^ets(d), 

=  ets{d)ty 
=  \x\xXy.Xy  y, 

=  XxXxXyXzi  . , .  Az„  .Zixy,  ...yZnXy. 


To  be  precise,  for  the  extracted  terms  of  the  induction  axioms  we  need  simul¬ 
taneous  recursion  operators  in  case  r(A)  has  length  >  1.  These  can  be  de¬ 
fined  easily.  Furthermore  note  that  if  ets(d)  =  ri, . .  .,rjt  and  ets(e)  =  s,  then 
ets(d)ets(e)  =  ris, . . . ,  rjbs  and  A^ets(d)  =  A®  ri, . . . ,  A^r*.  In  the  last  clause 
the  (omitted)  types  are 


X 


p 


and 


p-^T{A)-^aj 

^3 
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where  t{B)  =  0*1 , . . . ,  o-n. 

The  following  can  be  proved  easily. 

Lemma.  FV(ets((i))  C  FV(rf)  U  €  FA(d)}. 

Lemma,  l^e  have  ets(d[t/x])  =  ets{d)[t/x]  and  ets{d[e/u])  =  ets(d)[ets(e)/^u]; 
and  if  d  and  e  have  the  same  normal  form  then  ets(d)  and  ets(e)  have  the  same 
normal  form  too. 

Soundness  Theorem.  Assume  that  to  any  assumption  variable  we  have 
assigned  a  list  and  a  new  assumption  variable  u:  mr  .A.  Relative  to  this 

assignment  we  can  find  for  any  derivation  d:  A  with  FA(d)  =  {t/i :  j4i  , . . . ,  w„:  } 

a  derivation 

p(d):  ets(d)  mr  A 

with  FA(p(d))  C  {ui'.xi  mr^i, . . . ,  Xn  mri4n}. 

Proof  Induction  on  d. 

As  an  example  we  compute  the  extracted  terms  of  the  derivation  Cases^^B  for 
case  splitting. 

ets(CasesA,jB)  =  Xy.zAftAyz 

where  if  :=  R{Xyi,zi.yi){\yi^zi.zi)  and  y,  z,  yi,zi  are  lists  of  variables  of  types 
p  :=  t{B).  Clearly 

if  true  fs  =  f,  if  false  fs  =  s. 

For  better  readability  we  use  for  iftAVS  the  notation 

if  A  then  r  else  s  fi. 

3  Proof  translation  and  the  direct  method 

As  is  well  known  a  proof  of  a  V3-theorem  with  a  quantifier-free  kernel  —  where 
3  is  viewed  as  defined  by  -iV-'  —  can  be  used  as  a  program.  We  describe  a  “direct 
method”  to  use  such  a  proof  as  a  program  (cf.  [9]),  and  compare  it  with  GodeFs 
negative  translation  followed  by  Harvey  Friedman’s  A-translation  (cf.  [2])  and 
the  program  extraction  described  above. 

3.1  The  direct  method 

Assume  we  have  a  classical  derivation  d:  Vx3y  B[x,  B  quantifier-free,  from 
closed  i7-assumptions  FA(d)  =  {vi:  Vxi  Ci, . . . ,  Cn}.  We  describe  an  al¬ 

gorithm  which,  applied  to  closed  terms  r,  returns  terms  s  such  that  B[r,  ^  holds 
provided  the  assumptions  vi, . . . ,  are  true.  The  algorithm  proceeds  in  three 
steps. 
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1.  Instanciate  d  to  r.  We  get  df:3yB[f,  i.e., 

df:  (V27.B[f,  ^  ±)  -►  1. 

2.  Apply  dr  to  a  fresh  assumption  variable  u:Vy.B[f,  ^  — >•  ±.  We  get 

dfu:  L. 

3.  Normalize  dfu:  ±.  From  its  normal  form  (rffu)J,  read  off  the  first  instance 

s  :=  |(dfu)i|. 

Below  we  describe  how  the  first  instance  |(dnz)i|  is  obtained. 

Clearly  we  may  assume  that  the  iJ-assumptions  ViiMxi  Ci  do  not  contain  A 
and  that  the  formula  5[r,  y\  is  of  the  form  Bi[^  where  the  Bi  do  not  contain 
A.  Therefore  the  fresh  assumption  u:^y.B[f^y\  — ►  X  in  step  2.  may  assumed  to 
be  u:  ^  . . .  — Bm[y\  — >•  X.  Finally  we  may  assume  that  d  contains  no 

free  object  variables.  If  it  does,  substitute  arbitrary  closed  terms  for  them. 

Let  d:X  (corresponding  to  (dfu)  1:1.  above)  be  a  normal  derivation  with 
FV(d)  =  0  of  X  from  assumptions 

u:  Vy.Bi[^  Bm[y\  ->  X, 

where  \fy.Bi[^  Bm[^  — >■  X  is  a  false  and  VCi, . . . ,  VCn  are  true  closed 

i7-formulas.  We  define  a  list  |d|  of  closed  terms,  called  the  first  instance  of  d, 
such  that  5i[|d|], . .  .yBm[\d\]  are  true.  |c/|  is  defined  by  induction  on  d.  Since  d 
is  normal  and  FV(d)  =  0  it  does  not  contain  axioms  (exept  the  truth  axiom, 
which  is  a  closed  i7-formulas  and  hence  may  be  assumed  to  be  among  the  77- 
assumptions  VCi).  To  see  this  recall  that  the  normal  form  of  any  closed  term  of 
type  nat  is  of  the  form  S(S(S  . . .  (50)  . . .))  and  of  any  closed  term  of  type  boole 
is  either  true  or  false;  hence  all  induction  zixioms  unfold.  Therefore  d  is  of  the 
form 

wsdi . .  .dk, 

where  s  are  closed  terms  and  di, . . . ,  djb  are  derivations  of  closed  quantifier-free 
formulas.  We  distinguish  two  cases. 

1.  di , . . . ,  djb  derive  only  true  formulas  (which  can  be  decided,  since  the  formulas 
are  quantifier-free  and  closed).  Then  w  cannot  be  one  of  the  Vi  since  all  VC,* 
are  true.  Hence  d  —  usdi .  ..dk  and  the  d,-  derive  Bi[s\,  So  let  |d|  :=  s. 

2.  There  is  a  minimal  i  such  that  d,*  derives  a  false  formula,  Ai  — >■ - »■  Am  — ► 

X  say.  Then  Ai  . . . ,  Am  are  true.  Without  loss  of  generality  we  may  assume 
that  di  =  e  where  e:  X  contains  assumptions  among 

w: Vy.Hi[^  Bm[^  X, 

vi:WCu...,Vn:\fCn, 

,  .  .  . ,  U)m  ■  Am  • 

Therefore  we  can  recursively  define  |d|  :=  |e|. 
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3.2  Proof  translation 

Now  we  describe  the  more  traditional  way  to  obtain  a  program  from  a  classical 
proof  of  a  formula  3?/ 5,  B  quantifier  free,  from  (not  necessarily  closed)  U- 
assumptions.  First,  stability  axioms  are  removed  by  applying  GodePs  negative 
translation.  We  obtain  an  intuitionistic  proof  of  3xB.  Since  intuitionistically 
-Nx-iB  is  equivalent  to  -t-3*xB  we  can  use  the  fact  that  intuitionistic  arith¬ 
metic  is  closed  under  Markov’s  rule  and  obtain  an  intuitionistic  proof  oi3*xB. 
For  the  latter  step  we  use  Friedman’s  yl-translation  from  [2]. 

In  principle  the  negative  translation  is  not  necessary  since  we  could  prove 
stability  for  all  atomic  formulas  by  case  splitting.  However,  these  proofs  would 
introduce  characteristic  functions  tp  which  might  lead  to  inefficient  programs. 

The  negative  translation  replaces  each  atomic  formula  Pby(P^±)-^± 
and  the  ^-translation  in  turn  replaces  P  by  PV*  A,  where  A  :=  3* a;  B.  So,  at  the 
end,  P  is  replaced  by  ((P  V*  A)  ^  A)  A  which  is  intuitionistically  equivalent 
to  (P  A.  Hence  we  merge  the  two  steps  and  define  the  A-iranslation 

B^  of  a  formula  B  to  be  obtained  by  replacing  any  atomic  subformula  P  of  P  by 
(P  -H-  A)  — ^  A  (including  P  =  X;  this  is,  of  course,  not  optimal  but  convenient 
for  comparison  with  the  direct  method).  A  similar  translation  for  first  order  logic 
due  to  Leivant  [5]  is  described  in  [10],  2.3.,  page  64,  Theorem  3.20  (i). 

Note  that  any  derivation  d  of  some  formula  B  from  assumptions  Ci, . . . ,  Cn 
becomes  after  the  A-translation  a  derivation  of  B^  from  Ci  .  To  see 

this  recall  that  our  logical  rules  are  those  of  minimal  logic  and  hence  give  no  extra 
treatment  to  falsity.  Also  the  axiom  schemes  (exept  the  truth  axiom,  the  falsity 
axiom  and  Efqatom)  which  will  be  viewed  as  /7-assumptions)  remain  instances 
of  the  same  axiom  scheme  after  the  A-translation.  E.g.  boolean  induction 

P[true/p]  P[false/jp]  VpP 

is  translated  into  . 

B^  [true/p]  B^  [false/p]  —»•  Vp  B^ , 

which  again  is  an  instance  of  boolean  induction. 

Let  us  look  at  what  happens  with  /7-assumptions  under  the  A-translation. 
Again  we  may  assume  that  all  formulas  considered  do  not  contain  A. 

Lemma  1  For  any  quantifier-free  formula  C  there  is  a  derivation  d:C  C^. 

Proof.  Induction  on  C.  Let  C  ~  Bi  Bm  — >■  R  with  an  atom  R.  We 

must  derive 

{B R)  ^  ^  {R^  A)  ^  A. 

So  assume 

u  :  B  R, 

Vi-.Bf, 
w  :  R^  A. 


We  must  show  A. 
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Case  for  some  i.  Let  Bi  =  Ci  — >■  Pi  with  atoms  Pi.  Then 

Vi-.di^  -*  (Pi  A)  A 

and  we  have 

eij  [u,“  ] :  =  Stabcij  .u^ \u^*  -Efqp.  {vuj ) :  Cij , 

Ci  [«,"] :  =  Xwf*  .uj  Xu^*  Wi :  ~*Pi . 

By  IH  we  have  dcijiCij  — >  C^.  Hence 

5i(dci,«ii)  •  •  •  i<iCint eimX^wp .Efq^(e,t«i)):  A. 

Case  uf :  Bi  for  all  i.  Then 

i4 

The  extracted  terms  for  this  derivation  are 

d^^^  =  Xxu...,Xm,z^  if  --Bi  then  xid^^l^  . .  else 

if  -~>Bm  then  ®ise 

z  fi . .  .fi, 

where  Xi,  z  are  the  lists  of  variables  associated  with  Vii  Bf,w. 

* 

Here  we  have  used  case  splitting  formulas  Bi. 

Up  to  now,  the  formula  A  could  have  been  arbitrarily  chosen.  If  we  want  to 
use  the  A-translation  to  extract  the  computational  content  from  a  classical  proof 
we  have  to  choose  a  particular  A  involving  the  strong  existential  quantifier. 

Lemma  2  Let  Bi[x^^  be  quantifier-free  formulas  without  A,  and 

A[xl  := 

* 

Then  we  can  find  a  derivation  of  {fiy.B\[x^  ^  Bm[^y  ^ 

Proof.  Let  y  be  given  and  assume  Bf  and  w:L^  A.  We  must  show  A. 

Case  :  -iB,-  for  some  i.  Let  Bi  =  Ci  Pi  with  atoms  Pi.  Then 

Vi-.Ci'' ^  {Pi  ^  A) A 

eij  [uj"  ] :  =  Stabcij  .uf  .Efqp,  {vuj ) :  Cij , 

ei ] :  =  Xwp  .uf  Xu^*Wi'.  ~>Pi. 


and  we  have 
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Using  dcij '  Cij  — »•  from  Lemma  1  we  obtain 

Case  uf :  Bi  for  all  i.  Then 

. «m>:^ 

Note  that  the  assumption  it;:  ±  ►  A[£]  has  not  been  used.  The  extracted 

terms  for  this  derivation  are 


=  Xy,xi,...,x, 


'cn---dc“„.0  else 


if  ->Bm  then  x^dcl,  ■  ■  else 

y 

where  x,-,  z  are  the  lists  of  variables  associated  with  Vi\Bf,w. 

Theorem  (Friedman).  For  any  derivation 

d[u:'iy.Bi[x,^  ^  Bm[x,y\  ±,i;i:VCi, . . . ,  v„:  VC„]:  X 
with  Bi,  Cj  quantifier-free  and  without  A  we  can  find  a  derivation 

: VCi , . . . .  : VC7„] : 3* Bi [x, y] . 

i 

Proof.  Let  :=  3*y  Bi[x,^  and  consider  the  ^[^-translation 
d^t^K,t;i,...,t;;]:(X-X[x])-.A[^ 

where 

u':  (Vy.Bi[x,  Bm[x,  y]  ->■  X)^ti]_ 

of  d,  obtained  by  just  changing  some  formulas.  By  Lemma  1  we  have 

d.,k:VQ]:(VCi)^'^. 

By  Lemma  2  (now  using  the  particular  choice  of  A[^)  the  A  [^-translation  of 
the  assumption  u  is  provable  without  assumptions: 

:  (Vy.Si  [x,  yl  [x,  y]  X)^t^ 

Substituting  dy.[vi\'iCi\  for  u-  and  du  for  u'  in  d^^^  and  applying  the  result  to 
Efq^[^j  we  obtain 

:  VCi . :  VC„] :  3*y Bi  [x,  y| , 

i 

where 

Having  obtained  a  proof  d}’^  of  an  existential  formula  Bi[x,  ^  we  can 

then  apply  the  general  method  of  extracting  terms  to  this  proof.  It  yields 

since  extracting  terms  commutes  with  substitution. 
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3.3  Comparison 

We  now  prove  that  the  value  of  the  extracted  terms  when  instanciated  with  a 
list  f  of  closed  terms  is  in  fact  the  same  as  the  result  of  the  direct  method. 

So  consider  again  the  situation  of  Friedman’s  Theorem,  i.e.  a  derivation 

X,tn:VCi,...,v„:VC'„]:± 

with  Bj,  Cj  quantifier-free  and  without  A.  We  just  observed  that  the  program 
(^tr)ets  extracted  from  the  translated  derivation  has  the  form  (1)  above.  Let  us 
try  to  understand  how  this  program  works.  First,  closely  follows  the 

‘structure  of  d.  The  reason  is  that  differs  from  d  only  with  respect  to  the 
formulas  affixed,  and  when  forming  the  extracted  terms  this  affects  only  the 
types  and  the  arities  of  the  lists  of  object  variables  associated  with  assumption 
variables. 

In  order  to  comprehend  d^f  and  d^^  let  us  have  a  second  look  at  the  proofs  of 
Lemma  1  and  2.  First  note  that  dwib*- VC,]:  (VC,')^t^  is  obtained  from  djiC,-  — >• 
constructed  in  the  proof  of  Lemma  1  by 

dvi  = 

Since  Vi  has  type  VC,-,  which  is  a  Harrop  formula,  we  have  —  Xyi  df^.  Now 
from  the  proof  of  Lemma  1  we  obtain 

df  ®  =  if  -iBi  then  xidg®^ . . .  d^®^^  0  else 

(2) 

if  ~>Bm  then  ^,„d^®  ...dS®  else 
z  fi...fi, 

where  C,-  =  J5i  — ►  . . .  — >  Bm  — >  R  with  Bi  =  C,-  — >  P,-  and  z  are  the  lists 
of  variables  associated  with  Vi,w.  Furthermore,  d^®.  are  the  extracted  terms  of 

derivations  dcij>  C,-j  — >  C^^^  constructed  by  previous  applications  of  Lemma  1. 
Similarly 

d®^®  =  Ay,  , . . . ,  if  then  •  -  •  ^Cim  ^ 

"  (3) 

if  ->Bm  then  x,„d^®  ...d^®  0  else 

'-'ml  ^mntn 

where  Bi  ~  0%  Pi  and  Xi^z  are  the  lists  of  variables  associated  with  Vi,w. 
Furthermore,  d^®.  are  the  extracted  terms  of  derivations  dc  '.Ca  — >  C.^^^  con- 
structed  by  previous  applications  of  Lemma  1. 

This  analysis  makes  it  possible  to  prove  that  the  value  of  the  extracted  terms 
when  instanciated  with  a  list  f  of  closed  terms  is  in  fact  the  same  as  the  result 
of  the  direct  method  to  read  off  the  first  instance  provided  by  the  instanciated 
derivation 

J[fi:  Vy. J5i  [f .  ^  Bm  [r ,  ^  -L,  :  VCi , . . . ,  :  VC„] :  X 

Below  we  will  show  the  following 
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Claim.  For  any  normal  derivation 

e[u:Vff.Bi[f,^  ^  XjVitVCi, . .  .,t;„:VC'n]:  -L 

with  FV(e)  =  0  we  have 

|e|  =  I(e^M)*‘»K‘1r7x],  .... <=]^. 

We  then  obtain  that  the  instanciation  of  the  extracted  terms  (1)  with  r  for 
X,  i.e. 

, <», . . . ,  C]0>7^]  =  <■  •  •  - 

has  as  its  value  the  list  of  closed  terms  which  is  the  first  instance  of  the  instan- 
ciated  derivation  i.e.  |c?[f/^||.  For  by  the  claim  we  have 

|d[r7x]i|  =  [((d[r75]i)^'‘lnC[7£],  •  •  • , 

=  [((4r7i]^t^r‘*)i[C[r7^.  <“ . 

=  Mr7l]^'^)*‘'[C[i7^.  <.  •  •  • .  Ool. 

since  normalization  commutes  with  A [f] -translation  and  the  formation  of  ex¬ 
tracted  terms. 

It  remains  to  prove  the  claim.  We  use  induction  on  e.  Since  e  is  normal,  it 
must  be  of  the  form  e  =  wsei . . .  with  w  E  {u,  . . . ,  Vn}- 

Case  1.  ei, . . . ,  ejt:  derive  only  true  formulas.  Then  w  =  u,  k  =  m  and  the  e* 
derive  Bi[f,  By  definition  |e|  :=  s.  Furthermore 

.... 


by  the  form  (3)  of  since  all  B,  [r,  I]  =  (B,*  — »•  Pi)[r,  ^  are  true. 

Case  2.  There  is  a  minimal  i  such  that  e*  derives  a  false  formula,  Bii[^  — ^ 
- ^  -L  say.  Then  BaK,  •  •  • ,  are  true.  Without  loss  of  gen¬ 
erality  we  may  assume  that  e,*  =  Ai/;f /  where  /:±  contains 
assumptions  among 

u:  Vy.Bi[f,  ^  ^  B,„[r,  ^  ±, 

^1*  Bji, . . . ,  Wm'.  Dim- 

Therefore  by  definition  |e|  =  |/|.  Furthermore,  using  the  notation  du)j[wj]: 
for  the  derivation  obtained  by  applying  Lemma  1  to  Dij ,  we  have 

(e^Mr‘»K“[r7C.---,C]0 

^  f  (e* •  •  JO  if  w  =  u 

\  "JO  if  «<  = 

=  C, .  •  • ,  ClC  •  "<(0  by  (3)  and  (2),  respectively 
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SO  the  claim  follows  from  the  IH. 

4  Refinements 

In  applications  it  will  be  important  to  produce  extracted  terms  with  as  few  as 
possible  case  distinctions,  and  also  that  the  case  distinctions  should  be  over  as 
simple  as  possible  boolean  terms.  The  following  example  will  show  that  such 
improvements  are  indeed  necessary. 

4.1  The  root  example 

Let  /:  nat  ^  nat  be  an  unbounded  function  with  /(O)  =0.  Then  we  can  prove 

Vn3m./(m)  <  n  <  f{m  +  1). 

If  e.g.  f{m)  =  rn?,  then  this  formula  expresses  the  existence  of  an  integer  square 
root  m  :=  [y/r^  for  any  n.  More  formally  we  can  prove 

Vn3m.-»n  <  /(m)  A  n  <  /(m  +  1)  (1) 

from  the  assumptions 

t;i:Vn-<n  <  /(O),  t;2:Vnn  <  /(fif(n)). 

Here  <nat-^nat-^booie  jg  characteristic  function  of  the  natural  ordering  of  the 
natural  numbers  and  r  <  s  denotes  atom(<rs).  We  expressed  /(m)  <  n  by 
-»n  <  /(m)  and  /(O)  =  0  by  Vn  -m  <  /(O)  to  keep  the  formal  proof  as  simple  as 
possible.  In  order  to  have  iJ-assumptions  we  had  to  express  the  unboundedness 
of  /  by  a  witnessing  function  g. 

Now  let  us  prove  (1).  Let  n  be  given  and  assume 

u:  Vm.-in  <  /(m)  — >  n  <  f{m  +  1)  — >•  i.. 

We  have  to  show  ±.  From  and  u  we  inductively  get  Vm-m  <  f{m).  For 
m  :=  g{n)  this  yields  a  contradiction  to  V2. 

The  derivation  term  corresponding  to  this  proof  is 

d  :=  -I- 

Now  let 

A  :=  <  /(m)  A  n  <  /(m  +  1). 

The  program  extracted  from  d  is 

=  (d^)"'®K'^<^df;]0:nat. 

is  the  same  as  d  exept  that  Ind,„^-,n </(,„)  has  to  be  replaced  by 

An  -Rnat,(nat-+nat)— i-nat-t-nat 
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(since  r((-.n  <  /(m))^)  =  r(((n  <  /(m)  A)  ^  A)  (± A)  A)  = 
(nat  — »  nat)  — »•  nat  — >  nat),  and  the  assumption  variables  u^vi^v^  in  d  have  to 
be  replaced  by  (unary  lists  of)  object  variables 

nat  — ►  [(nat  nat)  — nat  — >•  nat]  -h-  (nat  ^  nat)  — ^  nat  — »•  nat, 

Xvj :  nat  — >■  (nat  — >  nat)  — >■  nat  — >■  nat, 

2:^3:  nat  — »■  nat  nat. 

The  subprograms  df^®,  df/  (of  the  same  types  as  are  given  by 

=  Xm^xi,X2,k.  if  n  <  f{m)  then  xiidO  else 

if  "in  <  /(m  + 1)  then  X2O  else 
m  fifi, 

df^  =  Xn,xi,k.  if  -m  < /(O)  then  xiO  else 
k  fi, 

df/  =  An,ik  k. 

Hence  the  normal  form  of  (d*^^)®^®  is 

R  {Xxi.k.if  -in  < /(O)  then  a^iO  else  k  fi) 
{Xm,wi^W2ik. 

if  n  <  f{m)  then  u;iidO  else 
if  -in  <  /(m  + 1)  then  W2^  else 
m  fifi) 

(P(n)) 

{Xkk) 

0. 

Informally,  =  H{g{n),  Xk  k,  0)  where  H:  nat  ^  (nat  — >■  nat)  — »■  nat  — »>  nat 

is  such  that 


i7(0,  aJi,  Ar)  =  if  -m  < /(O)  then  xiO  else  k  fi 
jH(m+ l,a;i,  Ar)  =  if  n  < /(m)  then  iy(m,  id,  0)  else 
if  -in  <  /(m  +  1)  then  ariO  else 
m  fifi. 

This  program  is  correct,  but  it  is  unnecessarily  complicated.  We  will  now  describe 
a  refined  ^-translation  which  will  simplify  the  type  of  the  auxiliary  function  H 
as  well  as  its  if-then-else  structure.  The  type  reduction  will  be  achieved  by  not 
replacing  all  atoms  P  by  (P  — >  A)  — ►  A. 
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4.2  The  refined  treinslation 

Let  L  be  a  set  of  formulas.  In  our  applications  L  will  consist  of  the  quantifier- 
free  kernels  of  the  lemmas  VC*,  and  in  addition  of  the  formula  Bi[x,  ^  — >  . . .  — > 
-h,  if  our  goal  formula  is  /j^-  Bi[x,y\. 

The  set  of  L-critical  relation  symbols  is  the  smallest  set  satisfying  the  fol¬ 
lowing  condition. 

If  (Cl  — Pi)  {Cm  Pm)  — ^  P(^  is  a  positive  subformula  of  an 

L-formula,  and  if  for  some  i  Pj  =  X  or  Pj  =  Q(s)  for  some  L-critical  relation 
symbol  Q,  then  R  is  L-critical. 

We  now  define  an  i4-translation  relative  to  L: 


L^:=A 

(J3  -  C)^  :=  -*  C^. 


if  R  is  L-critical 
otherwise 


We  will  write  P  G  Cl  if  the  atom  P  is  of  the  form  R{t)  for  some  L-critical  R  or 
P  =  l. 

A  quantifier-free  formula  Bi  Bm  R  will  be  called  L-relevani  if 

R  E  Cl, 


Lemma  1*  For  any  B  G  Neg(L)  and  any  C  E  Pos(L)  we  can  find  the  following 
derivations, 

(i)  dc’.C  C^.  Let  C  =  B  R.  Then  in  case  R  G  Cl  we  need  gsi  for  Bi 
L-relevant  and  fsj  for  Bj  L~irrelevant,  and  in  case  R^Cl  we  need  f^. 

(ii)  oc,D‘{{C  D)  — ►  A)  — >  C^  for  C  L-relevant,  Let  C  =  B  R,  Then 
we  need  Casesfi  if  R^  L,  and  also  gsi  or  fsi  depending  on  whether  Bi  is 
L-relevant  or  not. 

(Hi)  fs'B^  —y  B  for  B  L-irrelevant,  Let  B  =  C  R,  Then  we  need  d^. 

(iv)  gB‘B^  — )■  (P  — »■  A)  “>  A  for  B  L-relevant,  Let  B  =  C  ^  R.  Then  we 
need  some  eci,D  for  Ci  L-relevant,  and  Casescj  as  well  as  dcj  for  Cj  L- 
irrelevant. 


Proof,  Simultaneously  by  induction  on  the  subformulas  of  L~formulas.  In  each 
case  we  will  also  calculate  the  extracted  terms,  i.e.,  we  define  by  simultaneous 
structural  recursion  the  terms  /g®  and  g^^, 

(i)  Let  C  =  B  ^  R.  Case  R  e  Cl,  R^  -L.  We  must  derive 

{§  -^  R)  ^  B^  {R-^  A)  A. 

So  assume  u:  B  R,  vr.  Bf  and  w:R~^  A.  We  must  show  A. 

Let  Bi  for  i  E  {1,...,^}  be  relevant  and  Bj  for  j  E  {fc  +  1, . . . ,  m}  be 
irrelevant.  Assume  UiiBi  and  note  that  fjVj'.Bj.  Then  from  u:Bx  — »•  ...  — >• 
Bk  — Pfc+i  Pm  —>•  P  and  w:  R  A  we  get  A.  Now  cancel  Bk, 
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yielding  Bk  A.  Using  the  IH  —►  {Bk  A)  A  and  the  assumption 

B^  we  get  A.  Repeating  this  procedure  we  finally  cancel  UiiBi,  yielding  Bi  — »•  A. 
Using  the  IH  :  B^  (Bi  A)  A  and  the  assumption  B^  we  get  A,  as 
required.  The  derivation  term  is 

dc  =  \u,Vi,,,.,Vm,W, 

gs,  vi  Auf  ‘ .  •  •  •  Auf  ‘ .«)  (««i . . .  wjfc  (/b^+i  Vfc+i) . . .  (/b„  u^)) 

and  we  get 

=  A£i, . . . ,  xt, -  •  ■) 

where  Xi,  z  are  the  lists  of  variables  associated  with  Vi,w, 

Case  R  =  L,  Similarly,  using  Efq^  instead  of  w:R  A,  Then  in  we 
leave  out  Xz  and  replace  the  z  in  the  kernel  by  0. 

Case  R  ^  Cl.  Then  all  Bi  are  irrelevant.  We  must  derive 

{B^R)-^B^-^  R. 

So  assume  u:  B  ^  R  and  B^.  Using  the  IH  fsi-Bf  Bi  we  obtain  B  and 
hence  R.  In  this  case  ~  e. 

(ii)  Case  R  with  E  Cl,  R  ^  X.  Then  R^  =  (R  A)  A,  and  we  must 
derive 

{{R  D)  A)  {R^  A)  A. 

So  assume  u:{R—^D)  A  and  v:  R  A.  Then  clearly  A  can  be  derived,  using 
CaseSjR  and  Efqp.  The  derivation  term  is 

eB,D  =  t;.Cases/ji;Awi^.‘wAuf  ,Efq^(tiUi) 

and  we  get 

e|j  p  =  Xx,  y.  if  R  then  y  else  x  fi. 

Case  R  with  R=  ±.  Then  R^  =  Ay  and  we  must  derive 

{{±-^D)-.A)-^A. 

But  this  clearly  can  be  done  using  Efq^,,  and  we  have  —  A££. 

Case  B  C.  We  must  derive 

[{{B  ~^C)-^D)-^A]-^B^  C^. 

So  assume  u:  ((B  C)  D)  ^  A  and  v:B^ .  We  must  show  C^.  First  note 
that  we  can  derive  {C  D)  A  from  our  assumptions,  using  the  fact  that  by 
IH  we  can  derive  B^  {B  ^  A)  A  (with  gs  oi  fs,  depending  on  whether 
B  is  relevant  or  not).  But  then  the  claim  follows,  since  by  IH 

ec,D:{{C-^D)-^A)-^C^. 
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(iii)  Let  B  =  C  R.  Since  B  is  irrelevant  we  have  R  ^  Cl-  Then  {C 

R)^  =  R.  We  must  derive 

{C^-^R)-^C^  R, 

But  this  is  easy,  using  the  IH  dd'  Ci  Cf.  Clearly  =  e. 

(iv)  Case  R  with  G  Cl,  ^  ±.  Then  R^  =  {R A)  A,  and  we  must 
derive 

{{R  ^  A) A) -^{R-^  A) A, 

which  is  trivial.  We  have  =  Xxx. 

Case  R  with  R=  X.  Then  R^  =  A,  and  we  must  derive 

A  — >  ( J_  — >  .A)  — y  Af 

which  again  is  trivial.  We  have  =  Xx,  y.  x. 

Case  C  B.  By  assumption  B  is  relevant.  We  must  derive 

(C^  _  B^)  -y  {{C  -^B)-yA)^A.  (2) 

So  assume  u:  — »•  B^  and  v:  (C  — >■  B)  — >■  A. 

We  first  consider  the  case  where  C  is  relevant.  Then  we  have  ec^B'dC  —y 
B)  —y  A)  —y  by  IH(ii),  hence  B^  (using  v  and  u).  By  IH(iv)  for  the  shorter 
formula  B  we  know  gs'  B^  (B  A)  -y-  A  and  hence  {B  A)  ^  A.  But 
B  ^  A  can  easily  be  derived  from  our  hypothesis  v:(C  ~y^  B)  —y  A  and  hence 
we  obtain  A,  as  required.  The  derivation  term  is 


gc^B  =  Xu,v.gB{u{ec,Bv))Xuf  .vXu2Ui 


and  we  get 

y^9T(^dc^By))y^ 

We  finally  consider  the  case  where  C  is  irrelevant.  The  derivation  now  uses 
Casesc;  so  it  suffices  to  first  derive  (2)  under  the  additional  hypothesis  C,  and 
then  derive  (2)  under  the  additional  hypothesis  -tC. 

So  assume  Since  by  IH(i)  dc'C  —y  C^  we  obtain  C^  and  hence  B^ 

(using  w:(7^  — >■  B^).  By  IH(iv)  for  the  shorter  formula  B  we  know  gB’B^ 

{B  ^  A)  A  and  hence  {B  ^  A)  A.  But  B  ^  A  can  easily  be  derived  from 
our  hypothesis  v:  {C  —y  B)  ^  A  and  hence  we  obtain  Aj  as  required. 

Now  assume  u~:-tC.  Then  by  ex-falso-quodlibet  we  obtain  C  B  and 
hence  A,  using  our  hypothesis  v:  (C  —y  B)  —y  A, 

The  derivation  term  is 

gc^B  =  Au,  v.Casesc{Xu'^ .gB{u{dcu'^))Xuf  .vXu2Ui){Xu~ .vXuq  .EfqB{u~U3)) 

and  we  get 


ffcls  = if  C?  then  else  ff  fl. 
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Lemma  2*  Lei  Bi  he  quantifier-free  formulas  and  A  :=  Then  we 

can  find  a  derivation  of 

involving  Cases^ .  and  :  Cij  for  relevant  Bi  =  Q  Pi  and  fsj :  Bf  -4 

Bj  for  irrelevant  Bj . 

Proof.  Let  y  be  given  and  assume  vy.  B^, . .  .,Vm:  B^.  We  must  show  A.  We 
may  assume  that  5,-  for  «  G  {1, . . . ,  be  relevant  and  Bj*  for  j  G  {Ar  +  1, . . . ,  m) 
be  irrelevant 

Case  uj :  -.B,-  for  some  i  G  k}.  Let  Bi  =  Ci  B,-  with  atoms  B,-, 

Then 


in  case  B,-  ^  J.  and 


in  case  B,-  =  ±.  We  have 


Vi'.Ci  -^(Pi-^A)^A 


ViiCi  -^A 


^ij  [^r ] •  =  Stabcij-  .uj -Efqp.  {vuj ) :  C,y , 
e,*  [t/r  ] :  =  \wp  Wi :  -iB,- . 

Using  dcij'  Cij  — >■  C-j  from  Lemma  1*  we  obtain  in  case  B,-  ^  ± 

Vi{dci,eii)  .  •  ^{dci^.eimXXwp  .Efq^{eiWi)):  A 
and  in  case  B,*  —  X 

Case  uf:  Bi  for  all  i  e  {1, . . . ,  *;}.  Then 

3+y(uJ' /Sk+i «t+i ,  •  •  • ,  /b„  Wm>: 

The  extracted  terms  for  this  derivation  are 

c?ets  =  Xy,xi^...,xm>  if  -iBi  then  xid^^^  . . .  d^^^^  0  else 


if  -iBjb  then  ^JbC?Cfci  •  • 
y 

where  £,•  is  the  list  of  variables  associated  with  v,-:  Bf. 

This  ^-translation  relative  to  L  simplifies  the  extracted  terms  a  lot.  First 
the  derivations  of  C  are  necessary  only  for  those  lemmas  VC  where 

C  involves  B-critical  relation  symbols  (for  otherwise  we  have  C^  =  C).  For 
the  other  lemmas  a  derivation  of  C  — >•  C^  provided  by  Lemma  1*  involves  in 
most  cases  only  very  few  case  distinctions.  Finally,  in  the  derivation  given  by 
Lemma  2*  of  the  .^-translation  of  our  false  assumption  /j^.  Bf  -4  A  direct  case 
distinctions  are  only  necessary  for  the  relevant  B,-;  for  the  other  Bj  we  have  a 
derivation  of  B^  — >•  Bi  by  Lemma  T. 
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4.3  Examples  refined 

Let  us  now  come  back  to  our  initial  example  and  study  the  effect  of  our  re¬ 
finements  there.  We  can  relativize  the  ^-translation  to  the  set  L  of  formulas 
consisting  of 

n  <  /(O)  ±, 

n  <  f{9{n))y 

(n  <  /(m)  -»>  1)  n  <  /(m  -|- 1)  ^  1. 

Since  no  positive  subformula  of  an  L-formula  is  an  implication  with  a  <-atom 
as  its  conclusion  there  are  no  L-critical  relation  symbols.  Hence  only  negations 
5  ^  -L  are  L-relevant. 

We  now  repeat  our  treatment  of  the  root  example,  based  on  the  A-translation 
relative  to  L  and  the  refined  Lemmas  1*  and  2* .  The  derivation  term  correspond¬ 
ing  to  the  informal  proof  is 

d  ;=  Ind„,-,„</(m)n(t)in)u(ff(n))(w2n):  -L- 

Now  let 

A  :=  3*m.-'n  <  /(m)  A  n  <  /(m  +  1). 

The  program  extracted  from  d  is 

(^tr)ets  ^ 

(^A)ets  jg  same  as  d  exept  that  Ind^^-,n</(m)  has  to  be  replaced  by 

An  i2nat,nat 

(since  r((n  <  /(m)  1)^)  =  r{n  <  /(m)  A)  =  nat),  and  the  assumption 

variables  u,vi  in  d  have  to  be  replaced  by  (unary  lists  of)  object  variables 

Xu:  nat  — »•  nat  nat, 

Xui:  nat  — >  nat, 

whereas  V2  has  to  be  replaced  by  the  empty  list.  The  subprograms  d^^.df^  (of 
the  same  types  as  Xu,  aivi)  are  given  by  (cf.  Lemmas  2*  and  1*) 

=  Am,x.  if  n  <  f{m)  then  x  else  m  fi, 

<®  =  AnO. 

Hence  the  normal  form  of  is 

K0(Am,x.  if  n  <  f{m)  then  x  else  m  ii)(g{n)). 

Informally,  =  h{g{n))  where  /i:nat  — >■  nat  is  such  that 

h{0)  =  0, 

h{m  +  1)  =  if  n  <  f{m)  then  h{m)  else  m  fi. 
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We  conclude  with  a  further  example  which  is  still  simple  but  exploits  the 
general  results  of  section  4.2  more  seriously. 

Let  p  be  a  decidable  property  of  finite  binary  trees.  We  consider  the  problem 
of  computing  from  a  tree  xq  with  pxo  another  tree  j/o  with  pyo  but  minimal,  i.e., 
for  no  proper  subtree  of  yo  property  p  holds.  Let  px  mean  that  p  holds  for  no 
proper  subtree  of  x.  p  can  be  defined  inductively  by 

vi'ps,  V2\^x^  y.-^px  -ip2/  -^px  -^py  p{x,  y) 

where  e  denotes  the  empty  tree  and  [x,  y)  denotes  the  tree  with  immediate 
subtrees  x  and  y.  We  assume  further 

V3:pxo 

and  try  to  prove  3y.py  A  py,  i.e.,  under  the  additional  assumption 

u:  "iy.py  — py  — >■  1 

we  have  to  prove  ±.  Of  course,  in  the  proof  we  may  use  induction  on  trees,  i.e., 
for  any  formula  B{x)  the  axiom 

Ind5(a:) :  B{e)  — >■  (Var,  y .  B{x)  — ►  B{y)  — >■  B{{x,  y)))  — >■  \/x  B{x). 

Informally,  from  u  and  V2  we  get  Va?,  y .  par  pp  p(a?,  y)  and  hence,  by  induc¬ 

tion,  using  vi,  'ixpx.  This  clearly  contradicts  u  and  vg. 

From  the  proof  term 

d  =  uxo(IndpVi(Ax,  y,  V2xy(uxwi)(uyw2)wiW2)xo)v3 

we  obtain  the  program  (letting  now  Wi  range  over  functions  from  trees  to  trees) 
(d^T"  =  dt^^Xo(Rdf;(Ax,  y,  ti;i,  u;2. 

where  and  the  still  have  to  be  calculated.  Looking  at  our  assumptions 
u  and  Vi  it  is  clear  that  only  p  is  relevant.  Hence,  using  the  recipe  given  in  the 
proof  of  Lemmas  1*  and  2*,  we  obtain 

dl^^  =  Ax,w.wx  and  df^  =  d^'/ =  id. 

The  calculation  of  d®^®  is  a  little  bit  more  involved.  We  have 


,  ^2  •  if  px  then  zi  else  2:2  fi 

^2 py  then  zx  else  2:2  fi 

^ets  _  ^ets  _ :  j 

9px  —  9py  — 


and 
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Hence 

df^  =  \zi,Z2,Wl,W2,Z. 

if  px  then  zi  else  if  py  then  z^  else  wi{w2z)  fill. 
Therefore  we  get 

(dtr)et8 

{Xx,y,wi,W2,z, 

if  px  then  wix  else 

if  py  then  W2y  else  wi{w2z)  fifi) 

Xq 

Xq. 

This  means  =  g{xQ,  a?o)  where 

g(e,z)=  z, 

g{{x,y),z)=  ii  px  then  g{x,x)  else 

if  py  then  g{y,y)  else  g{x,9{y,z))  fifi. 
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1  Introduction 

The  modern  digital  computer,  a  force  which  has  shaped  the  latter  part  of  the 
20-th  century,  can  trace  its  origins  back  to  work  in  mathematical  logic  con¬ 
cerning  the  formalization  of  concepts  such  as  proof  and  computable  function. 
Numerous  examples  support  this  assertion.  For  instance,  in  his  development 
of  the  universal  Turing  machine,  A.M.  Turing  seems  to  have  been  the  first, 
along  with  J.  von  Neumann,  to  understand  the  potential  of  memory-stored  pro¬ 
grams  executed  by  a  universal  computational  device.  Moreover,  certain  func¬ 
tion  classes  and  proof  systems  can  be  viewed  as  prototypes  of  programming 
languages,  the  Kleene  //--calculus  and  imperative  programming  (pascal,  c); 
resolution  (Gentzen  sequent  calculus)  and  logic  programming  (Prolog);  the 
Church-Kleene  A-calculus  and  functional  programming  (lisp);  the  Girard  sys¬ 
tem  F  (polymorphic  A-calculus)  and  polymorphic  functional  programming  (ml). 

One  recurring  theme  in  recursion  theory  is  that  of  a  function  algebra  — 
i.e.  a  smallest  class  of  functions  containing  certain  initial  functions  and  closed 
under  certain  operations.  In  1925,  as  a  technical  tool  in  his  claimed  sketch 
proof  of  the  continuum  hypothesis,  D.  Hilbert  [48]  defined  classes  of  higher  type 
functionals  by  recursion.  In  1928,  W.  Ackermann  [1]  furnished  a  proof  that  the 
diagonal  function  (paia^a)  of  Hilbert  [48],  a  variant  of  the  Ackermann  function, 
is  not  primitive  recursive.  In  1931,  K.  Godel  [35]  defined  the  primitive  recursive 
functions,  calling  them  “rekursive  Funktionen”,  and  used  them  to  arithmetize 
logical  syntax  via  Godel  numbers  for  his  incompleteness  theorem.  Generalizing 
Ackermann ’s  work,  in  1936  R.  Peter  [72]  defined  and  studied  the  ifc-foid  recursive 

^This  is  a  preliminary,  condensed  version  of  an  article  to  appear  in  the  Handbook  of  Re¬ 
cursion  Theory^  ed.  E.  GrifFor. 

^Reaearch  partially  supported  by  NSF  CCR-9102896  and  by  US-Czechoslovak  Science  and 
Technology  Program  Grant  93  025. 
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functions.  The  same  year  saw  the  introduction  of  the  fundamental  concepts  of 
Turing  machine  (A.M.  Turing  [88]),  A-calculus  (A.  Church  [15])  and  ^t-recursive 
functions  (S.C.  Kleene  [57]).  By  restricting  the  scheme  of  primitive  recursion  to 
allow  only  limited  summations  and  limited  products,  the  elementary  functions 
were  introduced  in  1943  by  L.  Kalmar  [55].  In  1953,  A.  Grzegorczyk  [39]  studied 
the  classes  obtained  by  closing  certain  fast  growing  “diagonal”  functions 
under  composition  and  hounded  primitive  recursion  or  bounded  minimization. 

H.  Scholz’s  1952  [79]  question  concerning  the  characterization  of  spectra 
{n  G  N  :  (3  model  M  of  n  elements) (M  |=  0)}  of  first  order  sentences  (f), 
shown  in  1974  by  N.  Jones  and  A.  Selman  [54]  to  equal  ntime(2^('^)),  was  the 
starting  point  for  J.H.  Bennett’s  work  [9]  in  1962.  Among  other  results,  Ben¬ 
nett  introduced  the  key  notions  of  positive  extended  rudimentary  and  extended 
rudimentary  (equivalent  to  the  notions  of  nondeterministic  polynomial  time  NP 
and  the  polynomial  time  hierarchy  ph),  characterized,  the  spectra  of  sentences 
of  higher  type  logic  as  exactly  the  Kalmar  elementary  sets,  and  proved  that 
rudimentary  coincides  with  Smullyan’s  notion  of  constructive  arithmetic  (those 
sets  definable  in  the  language  {0, 1,  +,  •,  <}  of  arithmetic  by  first  order  bounded 
quantifier  formulas).  Only  much  later  in  1976  did  C.  Wrathall  [92]  connect 
these  concepts  to  computer  science  by  proving  that  the  linear  time  hierarchy 
LTH  coincides  with  rudimentary,  hence  constructive  arithmetic,  sets.  In  1963 
R.  W.  Ritchie  [73]  proved  that  Grzegorczyk’s  class  is  the  collection  of  func¬ 
tions  computable  in  linear  space  on  a  Turing  machine.  In  1965,  A.  Cobham  [26] 
characterized  the  polynomial  time  computable  functions  as  the  smallest  func¬ 
tion  algebra  closed  under  Bennett’s  scheme  of  hounded  recursion  on  notation.^ 
These  arithmetization  techniques  led  to  a  host  of  characterizations  of  computa¬ 
tional  complexity  classes  by  machine-independent  function  algebras  in  the  work 
of  D.  B.  Thompson  [86]  in  1972  on  polynomial  space,  of  K.  Wagner  [89]  in  1979 
on  general  time  complexity  classes,  and  others.  Function  algebra  characteri¬ 
zations  of  parallel  complexity  classes  and  of  certain  boolean  circuit  complexity 
classes  were  given  more  recently  by  the  author  [25,  23]  and  B.  Allen  [2].  Higher 
type  analogues  of  certain  characterizations  were  given  in  1976  by  K.  Mehlhorn 
[65],  in  1991  by  S.  Cook  and  B.  Kapron  [56,  29]  for  sequential  computation,  and 
in  1993  by  the  author,  A.  Ignjatovic,  B.  Kapron  [20]  for  parallel  computation. 
Though  distinct,  the  arithmetization  techniques  of  function  algebras  are  related 
to  those  used  in  proving  numerous  results  like  (i)  NP  equals  generalized  first 
order  spectra  (R.  Fagin  [31]),  (ii)  characterization  of  complexity  classes  via  fi¬ 
nite  models  (program  of  definitional  complexity  theory  investigated  by  R.  Fagin 
[32],  N.  Immerman  [52,  53],  Y.  Gurevich  [40],  and  others). 

From  this  preceding  short  historical  overview,  it  clearly  emerges  that  func¬ 
tion  algebras  and  computation  models  are  intimately  related  as  the  software 
(class  of  programs)  and  hardware  (machine  model)  counterparts  of  each  other. 

3  According  to  [65],  later  in  1972  independently  K.  Weihrauch  proved  a  similar 
characterization. 
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Historically,  these  notions  are  among  the  central  concepts  of  recursion  theory, 
proof  theory  and  theoretical  computer  science.  Perhaps  this  is  the  reason  that 
led  K.  Godel  [36]  in  1975  to  claim  that  the  most  important  open  problem  in 
recursion  theory  is  the  classification  of  all  total  recursive  functions,  presumably 
in  a  hierarchy  of  function  algebras  deterniined  by  admitting  more  and  more 
complex  operations.  While  much  work  characterizing  ever  larger  subrecursive 
hierarchies  has  been  done  by  Buchholz,  Girard,  Sacks,  Schwichtenberg,  Schiitte, 
Taken ti,  Wainer  and  others,  in  this  paper  we  concentrate  principally  on  sub¬ 
classes  of  the  primitive  recursive  functions  and  their  relations  to  computational 
complexity. 

Historically,  Cobham’s  machine  independent  characterization  of  the  polyno¬ 
mial  time  computable  functions  was  the  start  of  modern  complexity  theory,  in¬ 
dicating  a  robust  and  mathematically  interesting  field.  As  outlined  in  section  5, 
current  work  on  type  2  and  higher  type  function  algebras  suggests  directions  for 
the  extension  of  complexity  theory  to  higher  type  computation. 

2  Machine  computation  models 

In  this  paper,  we  survey  a  selection  of  results  which  illustrate  the  arithmetizar 
tion  techniques  used  in  characterizing  certain  computation  models  by  function 
algebras.  For  reasons  of  space,  proofs  will  not  be  given.  Familiarity  with  basic 
complexity  theory  is  assumed.  In  particular,  we  assume  familiarity  with  the 
Turing  machine  model  (tm),  and  its  variants  (the  Turing  machine  with  random 
access  (TMra),  nondeterministic  Turing  machine  (ntm),  E^j-Turing  machine,  al¬ 
ternating  Turing  machine  (atm),  oracle  Turing  machine  (otm)),  and  with  the 
boolean  circuit  model,  and  parallel  random  access  machine  model. 


3  Turing  machines 

Definition  1  (Chandra,  Kozen,  Stockmeyer  [14])  A  Turing  machine  M  with 
random  access  (TM,.a)  is  given  by  a  finite  set  Q  of  states,  an  input  tape  having 
no  tape  head,  k  work  tapes,  an  index  query  tape  and  an  index  answer  tape. 
Except  for  the  input  tape,  all  other  tapes  have  a  tape  head.  M  contains  a 
distinguished  input  query  state  g/,  in  which  state  M  writes  onto  the  leftmost 
cell  of  the  index  answer  tape  that  symbol  which  appears  in  the  A;-th  input  tape 
cell,  where  ^  =  XIi<m  ‘  2*  is  the  integer  whose  binary  representation  is  given 
by  the  contents  of  the  query  index  tape.  Unlike  the  oracle  Turing  machine  the 
query  index  tape  is  not  automatically  erased  after  making  an  input  bit  query. 
A  logtime  TM^a  runs  in  time  0(log7i),  where  n  is  the  length  of  the  input. 

Convention  2  From  now  on,  unless  otherwise  indicated,  for  any  sublinear  run¬ 
time  r(7i)  =  o(n),  the  intended  Turing  machine  model  is  TMra,  while  for  run¬ 
times  T{n)  =  n(n),  the  intended  Turing  machine  model  is  the  conventional 
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TM.  This  convention  applies  to  deterministic,  nondeterministic,  and  alternating 
Turing  machines.  While  it  is  a  simple  exercise  to  show  that  PTIME  is  the  same 
class,  regardless  of  model,  it  appears  to  be  an  open  problem  to  determine  the 
relationship  between  DTiME(r(n))  on  TM  and  TM^a,  for  T{n)  =  Q{n). 

The  classes  dtime(T(7i)),  NTiME(r(n)),  DSPACE(5(n)),  NSPACE(5(n)),  ptime, 
etc.  are  defined  as  usual. 

Definition  3  Let  ALOGTIME  denote  ATlME(0(logn)),  and  ALiNTiMEra  be 
ATlME(0(n))  on  a  TM^a-  The  logtime  hierarchy  LH  [resp.  the  linear  time  hierar¬ 
chy  LTH,  resp.  the  polynomial  time  hierarchy  PH]  is  the  collection  of  languages 
L  C  for  which  L  is  accepted  by  an  ATM  in  time  O(logn)  [resp.  0(n), 
resp.  with  at  most  0(1)  alternations.  Sjb-TlME(r(n))  is  the  collection 

of  languages  accepted  by  an  atm  in  time  0{T{n))  with  at  most  k  alternations, 
beginning  with  an  existential  state. 

Definition  4  A  function  /(a;i , . . . ,  has  polynomial  growth  resp.  linear  growth 
resp.  logarithmic  growth  if 

|/(a;i,.,  .,a:n)|  =  0(  m^  |xj|^),for  some  k 


resp. 

\f{xi,,,,,Xn)\=0{m^  |Xi|) 

resp. 

|/(a:i,...,a:„)|  =  0(log'‘(  l^iD.for  some  k. 

The  graph  Gf  satisfies  Gf{xjy)  iff  /(®)  =  y.  The  bitgraph  Bf  satisfies  Bf{x,i) 
iff  the  i-th  bit  of  f{x)  is  1.  If  C  is  a  complexity  class,  then  TC  [resp.  LinTC  resp. 
LogTC]  is  the  class  of  functions  of  polynomial  [resp.  linear  resp.  logarithmic] 
growth  whose  bitgraph  belongs  to  C.  In  this  paper,  QC  will  abbreviate  LinTC. 
The  iteration  f^'^{x)  is  defined  by  induction  on  n:  —  x,  f^‘^'^^\x)  = 

/(/(”) (x)).  With  this  notation,  the  iteration  log^”^  x  should  not  be  confused 
with  the  power  log”  x  =  (logx)”. 

3.1  Concurrent  random  access  machine 

Emerging  around  1976-77  from  the  work  of  Goldschlager  [37, 38],  Fortune-Wyllie 
[33],  and  Shiloach-Vishkin  [84],  the  parallel  random  access  machine  (pram) 
provides  an  abstract  model  of  parallel  computer  for  algorithm  development. 

A  concurrent  random  access  machine  CRAM  has  a  sequence  of 

random  access  machines  which  operate  in  a  synchronous  fashion  in  parallel. 
Each  Ri  has  its  own  local  memory,  an  infinite  collection  of  registers,  each  of 
which  can  hold  an  arbitrary  non-negative  integer.  Global  memory  consists  of 
an  infinite  collection  of  registers  accessible  to  all  processors,  which  are  used  for 
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reading  the  input,  processor  message  passing,  and  output.  Global  registers  are 
designated  Aff ,  M|, . . .,  and  local  registers  by  Mq,  Mi, M2, ...  -  local  regis¬ 

ters  of  processor  Pi  might  be  denoted  Mj^o?  -^1,1  j ....  A  global  memory  register 
can  be  read  simultaneously  by  several  processors  {concurrent  read,  rather  than 
exclusive  read).  In  the  case  where  more  than  one  processor  may  attempt  to 
write  to  the  same  global  memory  register,  the  lowest  numbered  processor  suc¬ 
ceeds  {'priority  resolution  of  write  conflict  in  this  concurrent  write  rather  than 
exclusive  write  model). 

Instructions  are  as  follows. 


Mres 

= 

constant 

Mres 

= 

processor  number 

Mres 

= 

Mopi 

Mres 

= 

Mopi  -f-  Mop2 

Mres 

= 

•1 

1 

Mres 

M  SP{Mop\ ,  Mqp2  ) 

Mres 

= 

LSP{MopuMop2) 

Mres 

= 

*Mopi 

Mres 

= 

*Mres 

= 

Afopi 

^M^res 

= 

Mopi 

GOTO 

label 

GOTO 

label 

IF  Mopi  —  Mop2 

GOTO 

label 

IF  Mppi  ^  Mop2 

HALT 

Cutoff  subtraction  is  defined  by  x^y  =  x-y,  provided  that  x>y,  else  0.  The 
shift  operators  MSP  and  LSP  are  defined  by 

•  MSP{x,y)  =  lx/2^},  provided  that  y  <  |a:|,  otherwise  ‘B’, 

•  LSP(a:,2/)  ~  X  —  2^  '  (L^/2^J))  provided  that  y  <  |x|,  otherwise 

The  CRAM  model  is  due  to  N.  Immerman  [53],  though  there  slightly  different 
conventions  are  made. 

Instructions  with  concern  indirect  addressing.  The  instruction  Mres  = 
*Mopi  assigns  to  local  register  Mres  the  contents  of  local  register  with  address 
given  by  the  value  M^pi.  Similarly,  Mres  =  *^opi  Performs  an  indirect  read 
from  global  memory  into  local  memory. 
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3.2  Circuit  families 

A  directed  graph  G  is  given  by  a  set  y  =  {1, . . .  ,7n}  of  vertices  (or  nodes)  and 
a  set  C  y  X  y  of  edges.  A  circuit  Cn  is  a  labeled,  directed  acyclic  graph 
whose  nodes  of  in- degree  0  are  called  input  nodes  and  are  labeled  by  one  of 
0, 1, xi, . . . ,  Xn,  and  whose  nodes  v  of  in-degree  fc  >  0  are  called  gates  and  are 
labeled  by  a  fc-place  function  i{v)  from  a  basis  set  of  boolean  functions.  A  circuit 
has  a  unique  output  node  of  out-degree  0. 

Boolean  circuits  have  basis  A,  V,  -i,  where  A,  V  may  have  unbounded  fan- 
in  (as  described  below,  the  AC*'  [resp.  NC*']  model  concerns  unbounded  fan-in 
[resp.  fan-in  2]  boolean  circuits).  A  threshold  gate  TUkyn  outputs  1  if  at  least  k 
of  its  n  inputs  is  1.  A  modular  counting  gate  uODk^n  outputs  1  if  the  sum  of  its 
n  inputs  is  evenly  divisible  by  k.  A  parity  gate  ©„  outputs  1  if  an  even  number 
of  its  n  inputs  is  1. 

Definition  5  ([77],  [3])  For  A;  >  0,  AC**  [resp.  NC*']  is  the  class  of  languages  in 
LOGTiME-uniform  SiZEDEPTH(n^(^^, 0(log*' n))  over  the  boolean  basis,  where 
A,V  have  unbounded  fan-in  [resp.  fan-in  2],  and  NC  =  U^AC*'  =  Uj^NC^. 
ACC(A;)  is  the  class  of  languages  in  LOGTiME-uniform  SizeDepth(7i^^^\  0(1)) 
over  the  basis  A,  V,  -i,  MODjk,nj  where  A,V  have  unbounded  fan-in,  and  ACC  = 
Ufc>2ACc(A:).  TC°  is  the  class  of  languages  in  LOGTiME-uniform  SiZEDEPTH(n^(^),  0(1)) 
over  the  basis  THjb,n- 


4  Some  recursion  schemes 

Kleene’s  normal  form  theorem  [57]  states  that  for  each  recursive  (partial)  func¬ 
tion  /  there  is  an  index  e  for  which  f{x)  =  U{p>y[T{e,Xjy)  =  0]),  where 
T,  ?7  are  primitive  recursive.  The  proof  relies  on  arithmetizing  computations 
via  Godel  numbers,  a  technique  introduced  in  [35]  by  Godel,  and  with  which 
Turing  computable  functions  can  be  shown  equivalent  to  //-recursive  functions. 
Since  then,  there  have  been  a  number  of  arithmetizations  of  machine  models 
[57,  58,  9,  26,  73,  86,  89,  25,  23],  etc.  Key  to  all  of  these  results  is  the  avail¬ 
ability  in  a  function  algebra  .7^  of  a  conditional  function,  a  pairing  function, 
and  some  string  manipulating  functions,  in  order  to  show  that  the  function 
NEXT m{XjC)  =  d  belongs  to  T,  where  c,  d  encode  configurations  of  machine  M 
on  input  x  and  d  is  the  configuration  obtained  in  one  step  from  configuration  c. 

Definition  6  If  A'  is  a  set  of  functions  and  OP  is  a  collection  of  operations,  then 
[A';  op]  denotes  the  smallest  set  of  functions  containing  X  and  closed  under  the 
operations  of  OP.  The  set  [A';OP]  is  called  a  function  algebra.  If  .F  is  a  class 
of  functions,  then  F**  is  the  class  of  predicates  whose  characteristic  function 
belongs  to  F*. 

Definition  7  Let  F"  =  [/i, /2} •  •  • ; Oi, O2, . . .]  be  a  function  algebra.  Let  0 
denote  operator  and  fix  a  derivation  D  of  /  €  .F.  The  rank  rko^oif)  of 
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applications  of  O  in  the  derivation  D  of  f  e  T  is  defined  by  induction.  If  / 
is  an  initial  function  then  rkoMf)  =  0-  Suppose  that  /  is  defined 

by  application  of  operator  Oi  to  functions  where  rko^oigj)  =  '^j 

for  1  <  j  <  m.  If  «  =  io  then  rko,D{f)  =  1  +  max{ri, . . .  otherwise 
~  max{7-i, . .  The  0-rank  rkoif)  of  a  function  f  e  is  the 

minimum  of  rko,D{f)  over  all  derivations  D  of  f  in 

Operations  which  have  been  studied  in  the  literature  include  composition, 
primitive  recursion,  minimization,  and  their  variants  including  bounded  compo¬ 
sition,  bounded  recursion,  bounded  recursion  on  notation,  bounded  minimiza¬ 
tion,  simultaneous  recursion,  multiple  recursion,  course-of- values  recursion,  di¬ 
vide  and  conquer  recursion,  safe  and  tiered  recursion,  etc.^  Good  surveys  of 
function  algebras  include  the  monographs  by  H.  Rose  [75]  and  K.  Wagner  and 
G.  Wechsung  [90]  (chapters  2,  10). 

4.1  An  algebra  for  the  logtime  hierarchy  LH 

Definition  8  The  successor  function  5(3;)  =  x  1;  the  binary  successor  func¬ 
tions  soj  satisfy  so(^)  =  2-a;,  5i(a;)  =  2-a:-l-l;  the  n-place  projection  functions 
.  .yXn)  =  Xk]  I  denotes  the  collection  of  all  projection  functions. 

Definition  9  The  function  /  is  defined  by  composition  (cOMP)  from  functions 
5l » •  •  • )  Pm  if 

/(Xi, .  . .  =  h(pi(xi, . .  .  jXn)^ .  •  .  ^gmi^lj  •  •  •  }2Jn))' 

The  function  /  is  defined  by  primitive  recursion  (pr)  from  functions  p,  h  if 

/(0,p)  =  g(y}, 

f{x  +  l,y)  =  h{x,yj{x,^). 

The  collection  VK  of  primitive  recursive  functions  is  [0,7,5;  COMP,  pr]. 

Primitive  recursion  defines  f{x  +  1)  in  terms  of  f{x),  so  that  the  computa¬ 
tion  of  f{x)  requires  approximately  2l®l  many  steps,  an  exponential  number  in 
the  length  of  x.  To  define  smaller  complexity  classes  of  functions,  Bennett  [9] 
introduced  the  scheme  of  recursion  on  notation,  which  Cobham  [26]  later  used 
to  characterize  the  polynomial  time  computable  functions. 

Definition  10  Assume  that  ho{x,y),hi{x,^  <  1.  The  function  /  is  defined 
by  concatenation  recursion  on  notation  (crn)  from  p,/io,/ii  if 

/(o,^0  =  p(p) 

/(5o(a:),p)  =  ^ho{x,y){fix,y)),  ifx^O 
/(5i(ar),^  =  ^/ii(x,i7)(/(^»  iO)* 

^In  this  paper,  for  uniformity  of  notation,  a  number  of  operations  are  introduced  as  bounded 
instead  of  limited  operations.  For  example,  Grzegorczyk’s  schemes  of  limited  recursion  and 
limited  minimization  are  here  called  bounded  recursion  and  bounded  minimization. 
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This  scheme  can  be  written  in  the  abbreviated  form 

f{0,y)=9{'^ 

The  scheme  CRN  was  introduced  by  the  author  in  [25]  though  motivated  by 
a  similar  scheme  due  to  J.  Lind  [64].  If  concatenation  of  the  empty  string  is 
allowed,  or  if  the  condition  hi{x,fi)  <  1  is  dropped  (as  in  Lind’s  scheme),  then 
the  resulting  scheme  is  provably  stronger. 

Definition  11  The  length  of  x  in  binary  satisfies  |x|  =  \log2{x+l)'];  MOD2(a;)  = 
^  “  2  ■  LfJ;  the  function  BlT(i,a;)  =  M0d2([^J)  yields  the  coefficient  of  2"  in 
the  binary  representation  of  x;  the  smash  function  satisfies  xif^y  —  2l®l‘l*'L  The 
functions  MSP,  LSP,  msp,  Isp  satisfy  MSP(x,y)  =  LSP{x,y)  =  x  mod  2*', 
msp{x,y)  =  [a;/2l^'j,  lsp{x^y)  =  x  mod  21*^1.  The  algebra  Aq  is  defined  to  be 

[0,  /,  50,  Si ,  bit,  |a;l,  #;  COMP,  CRN]. 

Small  sequences  of  small  numbers  can  be  encoded  by  a  function  in  Aq. 

Proposition  12  (Clote  [17])  If  f  e  Ao  then  there  exists  g  e  Aq  such  that  for 
all  X, 

=  (/(O, y).  •  •  •  >  /(|a:|  -  1)  j?))- 

The  following  lemma,  together  with  the  sequence  encoding  machinery  of  Ao, 
allow  a  proof  of  Aq  =  .Flh. 

Lemma  13  For  every  >  1, 

.DTiMESPACE(/op^(n),/o^^“^/’”(n))  C  Aq. 

Moreover,  DSPACE(0(loglog(n)))  on  a  TM^a  is  contained  in  lh. 

Theorem  14  (P.  Clote)  .4o  =  .Flh. 

Theorem  15  (Clote-Takeuti  [23]) 

TC®  =  [0,  /,  50, 51 ,  la;l,  BIT,  X ,  #;  COMP,  CRN]. 

Remark  16  Theorem  14  was  first  obtained  by  combining  the  author’s  result 
[25]  that  Aq  equals  FO  definable  functions,  and  the  Barrington-Immerman- 
Straubing  result  [3]  that  FO  =  LH,  an  analogue  of  Bennett’s  Theorem  41.  The 
current  proof  is  direct,  influenced  by  A.  Woods’  presentation  in  [91],  and  sim¬ 
plifies  the  argument  of  [3]  by  using  Lemma  13. 
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4.2  Bounded  recursion  on  notation 

Definition  17  The  function  /  is  defined  by  hounded  recursion  on  notation 
(brn)  from  g,  ho,  hi,  6  if 

/(0,j0  = 

f{so{^),y)  =  f^o(x,yJ{x,y))iix:^0  , 
fMx),y)  =  hi(x,y,f{x,y)). 

Theorem  18  (  A.  Cobham  [26],  see  H.  Rose  [75]) 

Uptime  =  [0,  /,  so,  si ,  comp,  brn]. 

Using  the  same  techniques,  one  can  characterize  the  class  Qp  polynomial  time 
computable  functions  of  linear  growth  as  follows,  (recall  that  *  is  concatena¬ 
tion),  and  the  other  assertion  follows  by  alternate  functions  in  bounding  the 
recursion  on  notation. 

Theorem  19  (D.H.  Thompson  [86]) 

Uptime  =  [0,/,so,5i,*;  comp,  brn] 

=  [0, 7, 5o ,  Si ,  X ;  COMP,  brn]  . 

Definition  20  The  function  /  is  defined  from  functions  g,ho,hi,k  by  sharply 
bounded  recursion  on  notation^  (sbrn)  if 

/(O,^  =  g{fj 

/(«o(a;),iO  =  ho{x,y,f{x,^),  iixi^Q 
=  h,i(x,y,f(x,^), 

provided  that  f{x,y)  <  \k(x,y)\  for  all  x,y. 

In  [64],  J.  Lind  characterized  .Flogspace  functions  on  words  it;  G  S*  as  the 
smallest  class  of  functions  containing  the  initial  functions  c=  (characteristic 
function  of  equality),  ♦  (string  concatenation)  and  closed  under  the  operations 
of  explicit  transformation,  log  bounded  recursion  on  notation,  and  a  (provably 
stronger)  version  of  concatenation  on  notation.  An  arithmetic  version  of  Lind’s 
characterization  is  the  following. 

Theorem  21 

FLOGSPACE  =  [0, 7,  So ,  Si ,  |x| ,  bit,  #;  comp,  crn,  sbrn] 

=  [0, 7,  So,  Si ,  msp,  #;  comp,  CRN,  sbrn] . 


®In  [23],  this  scheme  was  denoted  B2RN. 
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The  first  statement  appeared  in  [24,  23]  and  the  second  can  be  proved  using 
similar  techniques. 

Recently,  function  algebras  have  been  found  for  small  parallel  complexity 
classes.  Consider  the  following  variants  of  recursion  on  notation. 

Definition  22  The  function  /  is  defined  by  k-bounded  recursion  on  notation 
{k  —  brn)  from  g,  ho,  hi  if 

/(0,f)  =  g{x) 

/(2n,f)  =  /io(n,f,/(n,f)),  if  n  ^  0 

/(2n+l,x)  =  hi(n,f,/(n,f)) 

provided  that  /(n,  x)  <  k  holds  for  all  n,  x. 

Definition  23  The  function  /  is  defined  by  weak  hounded  recursion  on  notation 
(wbrn)  from  h,  A;  if 

F(0,f)  =  g{x) 

F(2n,x)  =  ho(n,x,F(n,x)),  if  n  ^  0 

F(2n  +  l,f)  =  hi{n,x,F{n^x)) 
f{n,x)  =  F{\nlx) 

provided  that  F(n,x)  <  k{n,x)  holds  for  all  n,x. 

The  characterization  of  polynomial  size,  constant  depth  boolean  circuits  with 
parity  gates  (resp.  MOd6  gates)  uses  sequence  encoding  techniques  of  Aq  to¬ 
gether  with  logtime  hierarchy  analogues  of  work  of  Handley,  Paris,  Wilkie  [42]. 

Theorem  24  (Clote-Takeuti  [23]) 

(1)  ACC(2)  =  [0,/,So,5i,|x|,BIT,#;COMP,CRN,1-BRN] 

(2)  ACC(6)  =  [0,/,So,5l,k|,BIT,#;COMP,CRN,2-BRN] 

(3)  ACC(6)  =  [0,/,So,5i,|a:|,BIT,#;COMP,CRN,3-BRN]. 

The  following  characterization  of  Falogtime  uses  earlier  techniques  with  a 
formalization  of  Barrington’s  trick  [4]  of  expressing  boolean  connectives  AND, 
OR  by  polynomial  size  bounded  width  branching  programs. 

Theorem  25  (P.  Clote  [17]) 

FALOGTIME  =  [0,/,So,5i,la;l,BIT,#;COMP,CRN,4  -  brn]. 

Theorem  26  (P.  Clote  [25]) 

NC  =  [0,  J,  So,  5i ,  |a;| ,  BIT,  #;  COMP,  CRN,  wbrn] 

AC^  =  {/  €  NC  :  t/cwbrnC/)  < 
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It  should  be  mentioned  that  independently  and  at  about  the  same  time,  B.  Allen 
[2]  characterized  NC  by  a  function  algebra  using  a  form  of  divide  and  conquer 
recursion,  and  noticed  without  giving  details  that  over  a  basis  of  appropriate 
initial  functions,  NC  could  also  be  characterized  by  the  scheme  of  wbrn.®  A 
precise  statement  of  Allen’s  characterization  is  given  later  in  Theorem  53. 

Using  such  techniques,  two  characterizations  of  NC*  were  given  in  [25,  23]. 
Levels  of  a  natural  time-space  hierarchy  between  .Uptime  and  .Fpspace  were 
characterized  in  [16]. 

4.3  Bounded  recursion 

In  1953,  A.  Grzegorczyk  [39]  investigated  a  hierarchy  of  subclasses  of  primi¬ 
tive  recursive  functions,  defined  as  the  closure  of  certain  initial  functions  under 
composition  and  bounded  recursion. 

Definition  27  The  function  /  is  defined  by  hounded  recursion  (br)  from  func¬ 
tions  g,  h,k\i 

/(Oj?)  =  9{f} 

=  h{x,yj{x,y)) 

provided  that  f{x,f}  <  k{x,f}  holds  for  all  x,y. 

Definition  28  Let  max  be  the  binary  maximum  function.  Define  the  following 
principal  functions  fo{x)  =  5(x)  =  a;  +  1,  fi{x,y)  =  X’\-y,  f2{x,y)  =  (a;  -f- 1)  • 

{y  +  l)j  fsix)  =  2®,  and  forn  >  3  fn+i{x)  =  fn\l)i  where  the  iterates  of  a 
function  g  are  defined  by  g^^\x)  =  x  and  =  g{g^^(x)).  Let  Ef  denote 

[0, /, s, /; COMP, br]  and  denote  Efn+i. 

In  [39]  Grzegorczyk  proved  that  for  all  n  >  0,  E^  is  properly  contained  in 
by  demonstrating  that  fn+i  ^  E^.  Concerning  the  relational  classes,  he 
showed  that  for  n  >  2,  is  properly  contained  in  ,  and  asked  whether  E^  C 
El  C  El.  This  question  remains  open.  In  fact  LTH  C  El  and  El  =  LINSPACE, 
so  Grzegorczyk’s  question  is  related  to  the  yet  open  problem  whether  the  linear 
time  hierarchy  is  properly  contained  in  linear  space.  An  interesting  partial  result 
concerning  the  containment  of  the  first  two  relational  classes  is  the  following. 

Theorem  29  (A.  Bel’tyukov  [8])  For  s>lj  let  I3g{x)  =  max(l,rE  -H 
Then  for  s>l,  El  =  {El3s)^.  Additionally,  El  =  El  implies  El  =  El 

To  obtain  this  result,  Bel’tyukov  introduced  the  stack  register  machine,  a  ma¬ 
chine  model  capable  of  describing  (^/)*.  The  stack  register  machine,  a  variant 
of  the  successor  random  access  machine,  has  a  finite  number  of  stack  registers 
5o, . . . ,  together  with  a  work  register  W.  Branching  instructions 


®See  remark  at  bottom  of  p.  13  of  [2]. 
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if  p{x\,...,Xm)-q{x\,...,Xm)  then  /»  else  Ij 

allow  to  jump  to  different  instructions  Ii,Ij  depending  on  the  comparison  of  two 
polynomials  whose  variables  are  current  register  values.  Storage  instructions 

W  =  5i 

allow  a  value  to  be  saved  from  a  stack  register  to  the  work  register.  Incremental 
instructions 


Si  -  Si  +  1 

perform  the  only  computation,  and  have  a  side  effect  of  setting  to  0  all  Sj  for 
j  <  i.  A  program  is  a  finite  list  of  instructions,  where  for  each  i  there  is  at  most 
one  incremental  instruction  for  Si. 

Apart  from  characterizing  or  LIN  SPACE,  BePtyukov  chacterized  the  linear 
time  hierarchy  lth.  The  papers  of  Paris,  Wilkie  [71]  and  Handley,  Paris,  Wilkie 
[42]  study  counting  classes  between  LTH  and  LINSPACE  defined  by  stack  register 
machines.  Unpublished  work  of  Handley  [44,  43]  and  of  the  author  [18]  further 
study  the  effect  of  nondeterminism  for  this  model. 

The  following  characterization  of  LINSPACE  in  terms  of  the  Grzegorczyk  hi¬ 
erarchy  was  proved  by  R.W.  Ritchie  [73]. 

Theorem  30  .^LINSPACE  =  5^. 

Theorem  31  (D.B.  Thompson  [86]) 

.Fpspace  =  [0,  /,  s,  #;  COMP,  br]  =  [0,  /,  5,  max,  ;  COMP,  br]. 

Definition  32  Let  k  be  an  integer.  The  function  /  is  defined  by  k -hounded 
recursion  (k-BR)  from  functions  h,  k  if 

my)  =  9{y) 

=  h{x,yj{x,y)) 

provided  that  fix^y)  <  k  holds  for  all  x^y. 

The  following  characterization  results  from  the  method  of  proof  of  Barrington’s 
characterization  of  ALOGTIME  [4,  3],  arithmetization  techniques  of  this  paper, 
and  work  of  Chandra,  Kozen,  Stockmeyer  [14]  implying  that  ATlME(n^^^^)  = 
PSPACE.  In  [13],  J.  Cai  and  M.  Furst  give  a  related  characterization  of  pspace 
using  safe-storage  Turing  machines,  a  model  related  to  BePtyukov’s  earlier  stack 
register  machines. 

Theorem  33  (P.  Clote  [18])  For  k>  4, 

.FPSPACE  =  [0,  /,  5o,  Sly  \x\y  BIT,  #;  COMP,  CRN,  k-BR]. 
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In  [89]  K.  Wagner  extended  Ritchie’s  characterization  to  more  general  complex¬ 
ity  classes. 

Theorem  34  (K,  Wagner  [89])  Let  f  he  an  increasing  function  such  that  for 
some  r  >  1  and  for  all  hut  finitely  many  x,  it  is  the  case  that  f{x)  >  x^.  Let  T 
temporarily  denote  the  algebra  [|/(2”)|;  COMP].  Then 

DSPACE(.F)  =  [0,  /,  5,  max,  /;  COMP,  br]*  =  [0,  /,  5,  /2,  /;  COMP,  br]*. 

The  class  DTlMESPACE(n^(^\  0(7i))  of  simultaneous  polynomial  time  and 
linear  space  can  be  characterized  from  the  previous  theorem  by  taking  f{x)  = 
x‘^.  As  referenced  in  [90],  S.V.  Pakhomov  [70]  has  characterized  general  complex¬ 
ity  classes  DTiMESPACE(r,  5),  DTlME(r),  and  dspace(5')  for  suitable  classes 
S,  T  of  unary  functions. 

The  class  QL  =  DTlME(n  •  (logn)^(^))  of  quasilinear  time  was  studied  by 
C.P.  Schnorr  in  [78].  In  analogy,  let  quasilinear  space  be  the  class  DSPACE(n  • 
(logTi)^^^)).  Though  Theorem  34  characterizes  quasilinear  space  via  a  function 
algebra,  there  appears  to  be  no  known  function  algebra  for  quasilinear  time. 
In  [41]  Y.  Gurevich  and  S.  Shelah  studied  the  class  NLT  {nearly  linear  time) 
of  functions  computable  in  time  0{n  •  (logn)^(^))  on  a  random  access  Turing 
machine  RTM,  which  is  allowed  to  change  its  input  tape.  Gurevich  and  Shelah 
show  the  robustness  of  nlt  by  proving  the  equivalence  of  this  class  with  respect 
to  different  machine  models,  and  give  a  function  algebra  for  nlt.  Their  algebra, 
defined  over  words  from  a  finite  alphabet,  is  the  closure  under  composition  of 
certain  initial  functions  and  weak  iterates  of  certain  string  manipulating 

initial  functions. 


4.4  Bounded  minimization 

In  [39],  Grzegorczyk  considered  function  classes  defined  by  bounded  minimiza¬ 
tion. 


Definition  35  The  function  /  is  defined  by  bounded  minimization  (bmin)  from 
the  function  g,  denoted  f{x,y)  =  fii  <  x[g{i^^  =  0],  if 


min{i  <  x  :  g{i,y)  =  0}  if  (3j  <  x){g{i,  j?)  =  0) 
0  else. 


For  n  >  0,  define  -  [0, /,  s,  /„;  COMP,  bmin].'^ 

'^In  [75],  following  work  of  K.  Harrow  [46,  47],  for  n  >  3,  is  defined  to  be 

[0,/,s,x^,/n; COMP, bmin].  By  Theorem  41  and  Proposition  38,  the  exponential  can  be 
defined  as  fiz  <  9(x,y){G{x,y,z)  =  0),  where  g  is  obtained  from  fn  and  initial  functions 
by  composition,  and  G  is  the  characteristic  function  of  the  graph  of  exponentiation,  which 
belongs  to  For  this  reason,  exponentiation  is  not  included  in  our  definition  of  A4”. 
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The  idea  of  proof  of  the  following  is  simply  to  encode  via  sequence  numbers  a 
definition  by  bounded  primitive  recursion  and  apply  the  bounded  minimization 
operator. 

Theorem  36  (Grzegorczyk  [39],  Harrow  [47])  For  n  >  3,  =  Af”. 

In  the  literature,  the  algebra  RF  of  rudimentary  functions  is  sometimes  defined 

by 

RF  =  [0, /,  S, +,  X ;  COMP,  BMIN]. 

As  noticed  in  [47],  it  follows  from  J.  Robinson’s  [74]  bounded  quantifier  definition 
of  addition  from  successor  and  multiplication  that  A4^  =  RF.  As  is  well-known, 
there  is  a  close  relationship  between  (bounded)  minimization  and  (bounded) 
quantification.  Terms  in  the  first  order  language  of  0, of  arithmetic 
are  defined  inductively  by:  0  is  a  term;  Xo,a;i, . . .  are  terms;  if  t,t*  are  terms, 
then  s(t),  t  1'  and  t  •  t'  are  terms.  Atomic  formulas  are  of  the  form  t  =  t' 
and  t  <t\  where  t^t’  are  terms.  The  set  Aq  of  bounded  quantifier  formulas  is 
defined  inductively  by:  if  0  is  an  atomic  formula,  then  €  Aq;  if  ^  G  Aq  then 
(j)A9,  and  (l>\/0  belong  to  Aq;  if  G  Aq  and i  is  a  term,  then  (3a;  <  t)(j){xjt) 
and  (Va;  <  t)(f>{x^t)  belong  to  Aq. 

Definition  37  A  predicate  R  C  N*'  belongs  to  CA  {constructive  arithmetic),  a 
notion  due  to  R.  Smullyan,  if  there  is  (j>{x)  G  Aq  such  that  R{ai, . . .  ,Ofc)  holds 
iff  N  1=  0(ai,  ...,ak)-  A  function  f{x)  G  5CA  if  the  bitgraph  Bf  E  CA  and  /  is 
of  linear  growth.® 

The  following  theorem  is  proved  by  using  quantifier  elimination  for  Presburger 
arithmetic  to  show  the  equivalence  between  first  order  formulas  and  bounded 
formulas  in  a  richer  language  allowing  congruences. 

Theorem  38  (Harrow  [46])  Ml  equals  the  collection  of  Presburger  definable 
sets,  =  CA  and  =  ^CA. 

Theorem  39  (J.  Bennett  [9])  The  graph  G{X,y,z)  x^  =  z  of  exponenti¬ 
ation  is  in  constructive  arithmetic. 


Corollary  40  The  function  algebra  [0,  /,  5o,  Si,  |x|,  BIT;  COMP,  CRN]  is  contained 
in  M^. 

Theorem  41  (Bennett,  Wrathall)  LTH  =  CA. 

Corollary  42  Mi  =  LTH,  and  M  =  Turn. 


®In  the  literature,  especially  in  [71],  a  function  /  is  defined  to  be  Aq  if  its  graph  Gf  belongs 
to  and  /  is  of  linear  growth.  It  easily  follows  from  Corollary  40  that  /  G  Gca  /  G 


AS'. 
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Though  the  linear  time  hierarchy  equals  the  bounded  arithmetic  hierarchy, 
there  is  no  known  exact  level-by-level  result.  The  sharpest  result  we  know  is 
due  to  A.  Woods  [91]. 

If  r  is  a  class  of  first  order  formulas,  then  denotes  the  collection  of  pred¬ 
icates  definable  by  a  formula  in  T.  Let  I)o,m  denote  the  collection  of  bounded 
quantifier  formulas  of  the  form  (3xi  <  y){^X2  <  2/) . . .  {Qxm  <  y)(f>  where  (/>  is  a 
quantifier  free  formula  in  the  first  order  language  0, 1, 4-,  •,  <•  Thus  £o,o  is  the 
collection  of  quantifier  free  formulas. 

Theorem  43  (A.  Woods  [91])  For  m  >  1  C  -  time(0(7i)). 

By  Corollary  42  and  Theorem  30,  Ml  =  LTH  C  LINSPACE  =  While 
LINS  PACE  is  clearly  closed  under  counting^  this  may  not  be  the  case  for  lth. 
A  typical  open  question  is  whether  7r(a;)  6  where  7r(aj)  is  the  number 
of  primes  less  than  x.  In  [71,  42]  J.  Paris,  A.  Wilkie  and  later  W.  Handley 
studied  the  effect  of  adding  fc-bounded  recursion  to  lth.  Using  the  techniques 
of  Barrington,  Paris,  Wilkie  and  Handley,  together  with  those  of  this  paper,  the 
following  result  can  be  proved. 

Theorem  44  (P.  Clote  [18]) 

For  any  k>A,  ALINTIME^a  =  [0,/,s,-|-,  x;  COMP,  BMIN,  A:-br]. 

As  in  Corollary  42,  .Fph  can  similarly  be  characterized. 

Theorem  45  (Folklore) 

=  [0,/,s,+,  x,#;comp,bmin] 

=  [0,/,S,+,  X,#;C0MP,BRN,BMIN]. 

The  second  assertion  of  the  last  theorem  was  sharpened  by  S.  Bellantoni  as 
follows.  Let  Ef  denote  E*  —  TlME(n^^^)).  Following  S.  Buss  [11]  let  nf  denote 
the  class  of  functions  computed  in  polynomial  time  on  a  Turing  machine  with 
oracle  A,  for  some  set  A  G  Ef .  With  this  notation,  Fm  =  UiUf. 

Theorem  46  (S.  Bellantoni  [6])  For  i>0, 

□f  =  {/  e  [0,/,ao.«l,.#;COMP,BRN,BMIN]  ;  rfcBMIN(/)  <  *}• 

4.5  Divide  and  conquer,  course-of-values  and  misc. 

Definition  47  Let  IZk  be  the  smallest  class  of  functions  definable  from  the 
constant  functions  0, . . . ,  the  projections  7,  the  characteristic  functions  of  the 
graphs  of  x,=  and  closed  under  composition  and  bounded  recursion. 

The  following  result  was  proved  by  the  Paris- Wilkie  modification  of  Bel’tyukov’s 
stack  register  machines. 
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Theorem  48  (Paris,  Wilkie  [71])  =  (Us)*- 

The  next  theorem  follows  from  the  author’s  work  in  [18]  and  is  based  on  Bar¬ 
rington’s  trick. 

Theorem  49  For  n  >  4, 

(7^n)♦  =  =  ALINTIMEra. 

In  [61],  Kutylowski  considered  oracle  versions  of  the  Paris- Wilkie  work. 
Definition  50  (M.  Kutylowski  [61])  /  is  a  fc-function®  if  for  all  Xi, . . . , 


/(xi,--*,Xn)  =  /(mm(xi,fe), •  •  •  ,mm(xn,A;))  <  k. 

For  a  family  T  of  functions,  Wk{F)  is  the  smallest  class  of  functions  containing 
7,  all  A;-functions  and  closed  under  composition  and  fc-bounded  recursion. 
The  function  /  is  defined  from  h  by  m-counting  if 

f{0,x]=gix) 

f{n  +  1,  x)  =  (/(n,  x)  -f  /i(n,  x))  mod  m 

The  class  CyVk(F)  is  the  smallest  class  of  functions  containing  /,  F,  all  m- 
functions  for  m  6  N  and  closed  under  composition,  /c-bounded  recursion  and 
arbitrary  counting. 

Theorem  51  (M.  Kutylowski  [61])  For  every  class  F  of  functions,  >V2(7^)*  = 
Wz{F)i^.  For  every  k  >  3,  there  exists  a  family  F  of  functions,  such  that 
Wk{F)i^  C  Wk-\-i{F)i^.  For  every  k  >3,  there  is  a  family  F  of  functions  such 
thatCyVk{F)^ceWk+i{F,). 

Parallel  algorithms  often  employ  a  divide  and  conquer  strategy.  B.  Allen  [2] 
formalized  this  approach  to  characterize  NC. 

Definition  52  The  front  half  Fli{x)  is  defined  by  MSP(x,  [|x|/2j)  and  the  back 
halfBu{x)  by  LSP(x,  [|x|/2j).  The  function  /  is  defined  by  polynomially  bounded 
branching  recursion  (pbbr)  from  functions  g,h  if  there  exists  a  polynomial  p 
such  that 


/(o.y)  =  9{0,^ 

/(I,j7)  =  s(l,jO 

f{x,^  =  y  >  1 

^What  is  here  called  a  fe-function  is  called  a  fc  -|-  1-function  in  [61].  As  our  definition  of 
fc-bounded  recursion  corresponds  to  Kutylowski ’s  definition  of  fe  -f  1-bounded  recursion,  the 
indices  of  Wk{T)  and  CWk{^)  differ  by  1  from  [61]. 
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provided  that  \f{x,yu, .  .,ym)\  <  p(max(|x|,  |^i|, . . . ,  |2/m|))  for  all  a;,  2/1, ,  2/^. 
Let  Seq{x)  =  0  if  a;  encodes  a  sequence^®  else  0.  If  x  encodes  a  sequence 
(xi, . . .  ,x„)  and  /  is  a  one-place  function,  then  the  operation  MAP  (similar  to 
CRN)  is  defined  by  MAP(/,a:)  =  (/(a^i), .  • . ,  fi^n))-  Define  the  bounded  shift  left 
function  by  SHL(a:,  i,  y)  =  a;  • 

Theorem  53  (B.  Allen  [2])  NC  is  characterized  by  the  function  algebra 

[0,  /,  5,  +,  |a;|,  BIT,  cond,  c<,Seq,  MSP,  SHL;  COMP,  MAP,  pbbr]. 

Allen  explicitly  did  not  attempt  to  find  the  smallest  set  of  initial  functions,  but 
went  on  to  develop  a  proof  theory  for  NC  functions,  similar  in  spirit  to  that  of 
S.  Buss  [11]  (see  also  [22]  for  related  work). 

The  Fibonacci  sequence  1, 1, 2, 3, 5, 8, . . .  is  defined  by  Fib{0)  =  Fib{l)  =  1, 
and  Fib{n  -{-  2)  =  Fib{n)  +  Fib{n  1).  This  is  a  special  case  of  course-of- values 
recursion. 

Definition  54  The  function  /  is  defined  from  functions  p,  h  by  course- of -values 
recursion  (vr)  if 


/(O.sO  =  g{f) 

+  =  h{x,f,(f{0,^,...,f{x,^)). 

The  class  VIZ  of  primitive  recursive  functions  is  easily  seen  to  be  closed 
under  VR.  For  complexity  classes,  it  is  of  more  interest  to  consider  a  bounded 
version  of  course-of-values  recursion,  where  f{x  +  1)  depends  on  at  most  two 
previously  defined  values  of  /. 

Definition  55  The  function  /  is  defined  from  functions  fc  by  bounded 

2-value  recursion  (bvr)  if 

/(o.jT)  =  giy) 

f{x  +  l,y)  =  h{x,^,f{x,f),f{r{x,f),fi) 

provided  that  /(x,^  <  k{x^y)  and  r{x,y)  <  x  for  all  x^y. 

Theorem  56  (Monien  [66])  Let  /2(x,2/)  =  (x  +  1)  •  (y  +  1),  and  etime  be 
Uc>iDTIME(2‘'‘”).  Then 

{/  €  ETIME  :  /  has  linear  growth  rate}  =  [0, /,  s,  /2;  COMP,  bvr]. 

When  a  pairing  function  is  available,  forms  of  simultaneous  recursion  can  usually 
be  deduced  from  corresponding  forms  of  non-simultaneous  recursion.  A  more 
powerful  version  of  simultaneous  recursion  was  introduced  in  [56]. 


Allen  [2]  uses  a  different  sequence  encoding  technique. 
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Definition  57  The  functions  /i, . . . , /n  are  defined  from  functions  pi, . . . 
hj, . . . ,  h® ,  /ij, . . . ,  /ijj  and  fci, . . . ,  by  multiple  bounded  recursion  on  notation 
if  the  fi  are  defined  by  simultaneous  recursion  on  notation  from  gl,  /ij,  h}  and 
moreover 


fi(x,y)  < 

fi{x,y)  <  ki{x,yJi{x,y),...Ji^i{x,y)),  for  2  <  i  <  n. 

The  following  non-trivial  closure  property  has  an  important  application  in  the 
Kapron-Cook  characterization  of  type  2  polynomial  time  computations  described 
in  the  next  section. 

Theorem  58  (Kapron-Cook  [56])  The  Cohham  algebra  [0, Z,  sq, Si,  #;  COMP,  brn] 
is  closed  under  multiple  bounded  recursion  on  notation. 

The  following  definition  and  theorem  will  be  used  in  the  next  section  to  char¬ 
acterize  the  type  2  parallel  complexity  class  NC. 

Definition  59  The  functions  /i are  defined  from  g,h^,h^,k  by  multiple 
weak  bounded  recursion  on  notation  if  /i(ic,  p)  =  Z^i(|a^|,  p),  where  Fi, . . . ,  are 
defined  by  multiple  bounded  recursion  on  notation  from  p,  h^,h^^k. 

Recall  that  the  algebra  A  =  [0, Z,  so,  si,  bit,  #;  comp,  CRN,  wbrn]  coincides 
with  the  parallel  complexity  class  NC,  consisting  of  those  functions  computable 
in  polylogarithmic  time  with  a  polynomial  number  of  processors  on  a  concurrent 
random  access  machine.  The  following  was  proved  by  the  author  and  will  appear 
in  the  journal  version  [19]  of  [20]. 

Theorem  60  The  algebra  A  is  closed  under  multiple  weak  bounded  recursion 
on  notation. 

4.6  Safe  recursion 

All  the  function  algebras  from  the  previous  subsection  are  defined  from  spe¬ 
cific  initial  functions,  using  some  version  of  bounded  recursion.  Without  any 
bound,  even  schemes  such  as  WBRN  can  generate  all  the  primitive  recursive 
functions.  Recently,  certain  unbounded  recursion  schemes  have  been  intro¬ 
duced  which  distinguish  between  variables  as  to  their  position  in  a  function 
f{xiy. . . ,  pi, . . .  ,Pm)*  Variables  Xi  occurring  to  the  left  of  the  semi-colon 
are  called  normal,  while  variables  yj  to  the  right  are  called  safe.  By  allowing 
only  recursions  of  a  certain  form,  which  distinguish  between  normal  and  safe 
variables,  particular  complexity  classes  can  be  characterized.  Normal  values  are 
considered  as  known  in  totality,  while  safe  values  are  those  obtained  by  im- 
predicative  means  (i.e.  via  recursion).  Sometimes,  to  help  distinguish  normal 
from  safe  positions,  the  letters  u, v, w, x,y,z,...  denote  normal  variables,  while 
a,6,c, ...  denote  safe  variables.  This  terminology,  due  to  Bellantoni-Cook  [7], 
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was  chosen  to  indicate  that  a  safe  position  is  one  where  it  is  safe  to  substitute  an 
impredicative  value.  Related  tiering  notions,  though  technically  different,  have 
occurred  in  the  literature,  as  in  [211  and  most  especially  in  work  of  D.  Leivant 
(see  [62]). 

If  T  and  O  are  collections  of  initial  functions  and  operations  which  distin¬ 
guish  normal  and  safe  variables,  then  NORMAL  fl  [f\0\  denotes  the  collection 
of  all  functions  f{x\)  G  [T\0]  which  have  only  normal  variables.  Similarly, 
(normal  n  [T\0])^  denotes  the  collection  of  predicates  whose  characteristic 
function  f{x\)  has  only  normal  variables  and  belongs  to  {T\0]. 

Definition  61  (Bellantoni-Cook  [7])  The  function  /  is  defined  by  safe  compo¬ 
sition  (scoMP)  from  ^f,  wi, . . . ,  vi , . . . ,  Vm  if 

f(x;  a)  =  g{ni{£:,), w„(x;);  Vi  (x;  S),...,  Vm(x;  S)). 


If  h(x;y)  is  defined,  then  SCOMP  allows  one  to  define 

f{x,y,)  =  h{ll^{x,y,y,I^’''{x,y;))  =  h{x;y). 

However,  one  cannot  similarly  define  g{]x,y)  =  h{x]y). 

Definition  62  The  function  /  is  defined  by  safe  recursion  on  notation^^  (srn) 
from  the  functions  if 


fiO,y;a)  =  g{y,S) 

/(«o(®),jr;o)  =  h(,{x,y,a,f{x,y;a)),  provided  x  #  0 
f(si(x),y;S)  =  hi{x,y,a,f{x,y;a)). 

Define  the  following  initial  functions  by 


(0-ary  constant) 

(projections) 

(successors) 
(binary  predecessor) 

(conditional) 


aj-n  nn<j< 


n 

n  +  m 


5o(;a)  =  2  •  a,  5i(;a)  =  2  •  a  +  1 
P(;a)=La/2J 

h  if  a  mod  2  =  0 
c  else. 


-  V? 

C'(;o,6,c)  =  I 


The  function  algebra  B  is  defined  by 


[0, 7, 5o,  ^i,  F,  C;  SCOMP,  srn). 


[7]  this  scheme  is  called  pTedicaiive  notational  recursion. 


Theorem  63  (Bellantoni-Cook  [7])  The  polynomial  time  computable  functions 
are  exactly  those  functions  of  B  having  only  normal  arguments,  i.e. 


Uptime  =  normal  n  B. 

This  approach  has  led  to  other  characterizations  of  familiar  complexity 
classes  using  safe  variants  of  unbounded  recursion  schemes. 

Theorem  64  (Bellantoni  [5]) 

{/  €  .Flogspace  :  |/(x)|  =  0(log|a;|)}  =  {f{x\) :  /  €  [0,/,5i,P,C;scomp,srn]} . 
Corollary  65 

LOGSPACE  =  (normal  n  [0,J,5i,P,C';SCOMP,SRN])*. 

Definition  66  The  function  /  is  defined  by  safe  minimization  (smin)  from  the 
function  g,  denoted  f{x',  b)  =  p>a[g{x;a,b)  mod  2  =  0)],  if 


/(x;  b)  = 


min{a  :  g{x;  a,  6)  =  0},  if  such  exists, 
0  else. 


The  algebra  fiB  =  [Q,  /,  ,  P,  C]  SCOMP,  SRN,  smin]  .  Let  fiBi  denote  the  set 

of  functions  derivable  in  fiB  using  at  most  i  applications  of  safe  minimization. 

Theorem  67  (Bellantoni  [6]) 

af  =  {f{S;)  :  f  €  fiBi]  . 

Definition  68  (Bellantoni  [5])  The  function  /  is  defined  by  safe  recursion^^ 
(sr)  from  the  functions  g,  h  if 

fiO,y;S)  =g(f,a) 

/(x  +  1,  jf;  o)  =  h{x,  f,S,f{x,f,  a)). 


Define  the  following  initial  functions  by 


(successor) 

(predecessor) 


S^(;  ft)  —  ft  +  1 
Pr(;ft)  =  ft  -  1 


(conditional)  K{;a,b,c)  = 


b  if  ft  =  0 
c  else. 


Theorem  69  (Bellantoni  [5]) 


=  NORMAL  n  [0,  /,  5,  Pr,  K]  SCOMP,  sr]. 
[5]  this  scheme  is  called  predicative  primitive  recursion. 
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W.  Handley  (unpublished)  independently  obtained  Theorem  69. 

Turning  to  parallel  computation,  by  building  on  Theorem  26,  S.  Bellan- 
toni  [5]  characterizes  NC  as  those  functions  with  normal  variables  in  an  algebra 
built  up  from  0,  /,  5'o,  ^i,  the  conditional  C,  the  bit  function  bit,  the  length 
function  L{;a)  =  |a|,  a  variant  #'  of  the  smash  function,  and  closed  under 
safe  composition,  concatenation  recursion  on  notation  and  a  version  of  safe 
version  of  weak  bounded  recursion  on  notation.  Define  the  half  function  by 
H{x)  =  La;/(2n®l/2l)J^  and  note  that  the  least  number  of  times  which  H  can  be 
iterated  on  x  before  reaching  0  is  ||a:||.  The  function  /  is  defined  by  safe  weak 
recursion  on  notationf^  from  the  functions  g^hii 


/(0,j?;a)  =  p(^;o) 

f{x,f,a)  =  a, a)),  provided  x  ^  0. 


Theorem  70  (S.  Bellantoni  [5]) 

NC  =  [0,  /,  ^0,  51,(7,  L,  BIT,  SCOMP,  CRN,  SWRN]. 

Following  [2],  define  BH(a;)  =  a;  mod  2n®l/2l  and  FH(a:)  =  msp(a;,  bh(x)). 
The  back  half  bh{x)  consists  of  the  r|a;|/2]  rightmost  bits  of  x^  while  the  front 
half  fh{x)  consists  of  the  [|x|/2j  leftmost  bits  of  x.  In  [10]  S.  Bloch  defines  two 
distinct  safe  versions  of  Allen’s  divide  and  conquer  recursion. 


Definition  71  (S.  Bloch  [10])  The  function  /  is  defined  by  safe  divide  and 
conquer  recursion  (sdcr)  from  the  functions  g,h\{  The  function  /  is  defined  by 
very  safe  divide  and  conquer  recursion  (vsdcr)  from  the  functions  g,  h  if 


f{x,y,z;a) 


g(x,z;a)  if  |a;|  <  max(|y|,l) 

h{]  X,  z,  a,  /(fh(;  ar),  y,  z\  a),  /(bh(;  x),  y,  z\  a))  else. 


Note  that  in  VSDCR  the  iteration  function  h  has  no  normal  parameters,  and 
hence  cannot  itself  be  defined  by  recursion. 


Theorem  72  (S.  Bloch  [10])  There  is  a  collection  BASE  of  initial  functions,  for 
which}''^ 

ALOGTIME  =  (normal  fl  [BASE;  SCOMP,  VSDCR])* 

DSPACE(log^^^^  n)  =  (normal  n  [base;  SCOMP,  SDCR])*. 

It  seems  clear  that  linear  time  on  multitape  Turing  machines  can  be  character¬ 
ized  using  appropriate  initial  functions  (sufficient  to  define  nextm )  and  closure 
under  safe  recursion  and  some  form  of  very  safe  recursion.  Details  have  been 
worked  out  by  S.  Bloch  in  unpublished  work,  and  a  related  category  theoretic 
characterization  has  been  announced  by  J.  Otto  [69]. 

^^In  [5]  this  scheme  is  called  log  recursion. 

Bloch  states  his  second  result  in  terms  of  polylogarithmic  parallel  time  (with  no  processor 
bound).  Since  parallel  time  equals  sequential  space,  this  is  an  equivalent  assertion. 
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5  Type  2  functionals 

Many  programming  languages  allow  functions  to  be  passed  as  parameters  to 
other  functions  or  procedures.  For  instance,  FORTRAN,  PASCAL,  and  C  allow 
function  parameters,  while  C'^'^  supports  function  templates  and  ADA,  ml  admit 
limited  polymorphism.^^  The  oracle  Turing  machine  is  a  reasonable  construct  to 
model  function  parameter  passing,  though  it  has  principally  been  used  to' study 
reducibilities  A  <t  B,  A  B  etc.  between  sets.  Nevertheless,  higher  type 
functional  complexity  theory  is  a  new  area  with  fundamental  open  problems. 
In  particular,  though  various  classes  have  been  been  proposed  as  candidates  for 
the  feasible  type  2  functionals,  there  is  not  yet  general  agreement  about  the 
right  notion.  For  reasons  of  space,  only  a  few  recent  directions  in  higher  type 
functional  complexity  will  be  presented.  For  more  information,  see  the  survey 
[28]  by  S.A.  Cook. 

Definition  73  A  type  2  functional  F  of  rank  is  a  total  mapping  from 
X  into  N. 

Definition  74  An  oracle  Turing  machine  (otm)  is  a  Turing  machine  M  which 
in  addition  to  read-only  input  tape,  distinguished  output  tape  and  finitely  many 
work  tapes,  has  an  oracle  query  tape  and  oracle  answer  tape,  both  one-way 
infinite,  for  each  function  input.  Additionally  M  has  a  special  oracle  query 
state  for  each  function  input. 

In  order  to  query  a  function  input  f  at  Xj  the  machine  M  takes  steps  to 
write  X  in  binary  on  the  oracle  query  tape.  When  the  oracle  query  tape  head 
is  in  its  leftmost  square,  M  enters  a  special  query  state.  In  the  next  step,  M 
erases  both  the  oracle  query  and  answer  tapes,  writes  the  function  value  f{x) 
in  binary  on  the  oracle  answer  tape,  and  leaves  the  oracle  query  and  answer 
tape  heads  in  their  leftmost  squares.  Upon  entering  the  oracle  query  state, 
there  seem  to  be  two  natural  measures  for  the  time  to  complete  the  function 
query  f{x).  The  unit  cost,  considered  by  Mehlhorn  [65],  charges  unit  time, 
while  the  function  length  cost,  considered  by  Constable  [27]  and  later  Kapron 
and  Cook  [56],  charges  max{l,  |/(a;)|}  time.  The  machine  M  computes  the 
rank{n, m)  functional  F{fi, . . . ,  /n, xi, . . . ,  Xm)  if  M  has  n  oracle  query  states, 
query  and  answer  tapes  corresponding  to  /i , . . . ,  and  if  M  outputs  the  integer 
F{fi  xi ,...,  Xm)  in  binary  on  the  output  tape,  when  started  in  its  initial 

state  qq  with  input  tape  BX1BX2B  •  •  •  BxmB. 

Definition  75  An  OTM  M  is  a  polynomial  time  oracle  Turing  machine  (potm) 
if  M  computes  a  total  rank{n,  m)  functional  F  and  there  is  a  polynomial  p  such 
that  for  all  input  /i , . . . ,  /n,  a:i , . . . ,  Xm  and  times  t 

t  <  p(|moa;({a;i, . . .  ,®m}  U  AM{f,S,t))\). 


Polymorphism  allows  function  and  procedures  to  abstract  over  data  types  —  e.g.  a  generic 
sorting  algorithm  for  any  data  type  having  a  comparison  function. 
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OPT  is  the  collection  of  type  2  functionals  computable  by  an  oracle  polynomial 
time  oracle  Turing  machine. 

Example  76 

(1)  F(/,a:)  =  max{f{y)  :  y  <  |x|}  belongs  to  OPT. 

(2)  G{fyX)  =  max{f{y) :  \y\  <  |a;|}  does  not  belong  to  OPT. 

(3)  H{f,x)  =  belongs  to  OPT. 

In  [65]  K.  Mehlhorn  extended  Cobham’s  function  algebra  to  type  2  function¬ 
als.  A  modern  presentation  of  Mehlhorn’s  definition  uses  the  following  schemes. 

Definition  77  (Townsend  [87]).  F  is  defined  from  fl",  Gi , . . . ,  Gm  by  functional 
composition  if  for  all  /,  x, 

F(/,f)  =  HilGyilx), . .  5). 

F  is  defined  from  G  by  expansion  if  for  all  fig^x^y, 

=  G{f,x). 

F  is  defined  from  G,  H,  K  by  limited  recursion  on  notation  (lrn)  if  for  all  /,  x,  y, 
F(fj,0)  =  G(/>) 

F{f,  S,  y)  =  H{f,  S,  y,  F{1  S,  [|j )),  if  y  7^  0 

provided  that  F(fjX,y)  <  K{f,x,y)  holds  for  all  f,x^y. 

Definition  78  (Townsend  [87],  Kapron,  Cook  [29])  Let  X  be  a  class  of  type 
2  functionals.  The  class  of  basic  feasible  functionals  defined  from  X,  denoted 
bff(X),  is  the  smallest  class  of  functionals  containing  X,  0,so,5i,i^,#  and  the 
application  functional  Ap,  defined  by  Ap{f,  x)  =  f{x),  and  which  is  closed  under 
functional  composition,  expansion,  and  lrn.  If  F  €  bff(X),  then  F  is  basic 
feasible  in  X.  The  class  bff  of  basic  feasible  functionals  is  bff(0). 

Theorem  79  (Mehlhorn  [65])  BFF  is  the  collection  of  type  2  functionals  com¬ 
putable  on  an  OTM  with  unit  cost,  where  the  runtime  on  input  /,  x  is  bounded 
by  |F(/,^)|  for  some  F  belonging  to  BFF. 

It  is  clear  that  OPT  contains  functionals  which  are  not  intuitively  feasible. 
In  particular,  substituting  the  polynomial  time  computable  function  Xy.y^  for 
/  in  H,  where  H{f,x)  =  above  yields  H(Xy.y^,x)  =  which  is 

not  a  poly  time  computable  type  1  function  (example  due  to  A.  Seth  [81]).  The 
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following  example,  due  to  S.  Cook,  provides  a  functional  which  belongs  to  OPT 
yet  not  to  BFF. 

Let  •<  quasi-order  N  x  N  by  length  first  difference]  i.e.  (a,  6)  :<  (c,  d)  iff 
\a\  <  \c\  or  (|a|  =  |c|  and  |6|  <  \d\).  Transfer  this  ordering  to  N  by  a  standard 
polynomial  time  pairing  function.  Define  the  rank{l,  0)  functional  Lhy  L{f)  = 
<  i)(f{j)  :<  /(O)]-  Note  that  :<  defines  a  quasi-well  ordering  on  N  x  N, 
so  L  is  well  defined. 

Theorem  80  (S.  Cook  [28])  The  functional  L  belongs  to  opt  yet  not  to  bff. 

In  [56],  Kapron  and  Cook  lift  Cobham’s  characterization  of  polynomial  time 
computable  functions  to  functionals  of  level  2.  To  state  their  result,  the  notion 
of  length  of  a  function  and  that  of  second  order  polynomial  must  be  introduced. 

Definition  81  The  length  |/|  of  one-place  function  /  is  itself  a  one-place  func¬ 
tion  defined  by 

|/|(n)  =  m^{|/(a:)|}. 

|x|<n 

Let  /i , . . . ,  /m  be  variable  ranging  over  and  xi , . . . ,  Xn  be  variables  ranging 

over  N.  The  collection  C  of  second  order  polynomials  P(/i, . . . ,  /m,  , . . . ,  a^n) 

is  defined  inductively  as  follows. 

(i)  for  any  integer  c,  c  G  C, 

(a)  for  every  1  <  i  <  n,  Xi  E  C, 

(Hi)  if  P,Q  e  C  then  P  -\-Q  eC  and  P  'Q  e  C, 

(iv)  if  P  E  C  then  fi{P)  E  C  foi  1  <  i  <  m. 

Theorem  82  (B.  Kapron  and  S.  Cook  [56])  bff  is  the  collection  of  functionals 

computable  in  time  P(|/i|, . . . ,  |/n|,  |xi|, . . . ,  |x„i|)  for 
some  second  order  polynomial  P  on  an  OTM  with  function  length  cost}^ 

The  oracle  concurrent  random  access  machine  (ocram),  introduced  in  [20] 
has  instructions  for  (i)  local  operations  —  addition,  cutoff  subtraction,  shift, 
(a)  global  and  local  indirect  reading  and  writing,  (Hi)  control  instructions 
—  GOTO,  conditional  GOTO  and  HALT,  (iv)  oracle  calls,  where  in  one  step,  all 
active  processors  simultaneously  can  retrieve 

k—i 

where  i,  j  are  current  values  of  local  registers,  and  xi  is  the  0,1  value  held  in  the 
i-th  oifacle  register.  The  formal  details  of  the  this  model  ensure  that  the  size  of 

^®In  [51],  A.  Ignjatovic  has  given  an  alternate  proof  theoretic  proof  of  Theorem  82  and 
moreover  has  shown  the  same  result  to  hold  for  unit  cost. 
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the  function  value  returned  in  any  oracle  call  will  be  bounded  by  the  product 
of  the  number  of  active  processors  and  the  total  computation  time. 

The  OCRAM  is  formally  defined  as  follows.  For  each  k-aiy  function  argument 
/,  there  are  k  infinite  collections  of  oracle  registers^  the  i-th  collection  labeled 
Mq M2 . . .,  for  1  <  i  <  A;.  As  with  global  memory,  in  the  event  of  a 
write  conflict  the  lowest  numbered  processor  succeeds  in  writing  to  an  oracle 
register.  Let  res  (result),  opO  (operand  0)  and  opl  (operand  1)  be  non-negative 
integers,  as  well  as  op2,op3,. . .  ,op(2A;). 

In  addition  to  the  instructions  for  the  CRAM,  the  OCRAM  has  instructions 
concerning  the  oracle  registers  and  oracle  calls. 

=0 
=  1 
=  0 
=  1 

=  f{[Mopl  ■  •  •  Mop2]\’>  [Mop3  •  •  •  Mop4]2}  •  •  •  ,  [Mop(2jfc_i)  *  *  •  Mop(2fe)]fc) 

The  notation  [Mop(2i_i) '  * '  ^op{2i)\i  denotes  the  integer  whose  binary  notation 
is  given  in  oracle  registers  through  Mf;* 

In  characterizing  AC^  in  the  non-oracle  case,  Stockmeyer  and  Vishkin  [85] 
require  a  polynomial  bound  p{n)  on  the  number  of  active  processors  on  inputs  of 
length  n.  With  the  above  definition  of  OCRAM  one  might  hope  to  characterize 
the  class  of  type  2  functionals  computable  in  constant  parallel  time  with  a 
second-order  polynomial  number  of  processors  as  exactly  the  type  2  functionals 
in  the  algebra  A^,  Using  the  definitions  given  so  far,  this  is  not  true.  To  rectify 
this  situation,  proceed  as  follows. 

Definition  83  For  every  OCRAM  M,  functions  /,p  and  integers  x^t  the  query 
set  Q{Mj  f,x^t,g)  is  defined  as 

{y  :  M  with  inputs  /,  x  queries  /  at  p  in  <t  steps,  where  for  each 
i  <  t  the  active  processors  are  those  with  index  0,...,p(i)-l}. 

Let  M  be  an  OCRAM,  P  a  functional  of  rank  (1,1),  f  a  function  and  x,  t  integers. 
If  Q  C  N  then  define  /q('u)  =  /(u)  if  w  g  Q,  otherwise  0.  Define  M  =  (M,  P) 
to  be  a.  fully  specified  OCRAM  if  for  all  /,  x,  t  the  OCRAM  M  on  input  /,  x  either  is 
halted  at  step  t  or  executes  at  step  t  with  active  processors  0, . . . ,  P(|/qJ,  |a;|)- 1 
where 

Q<  =  Q(M,/,x,t,P(|/Q,_J,|x|)) 
is  the  collection  of  queries  made  by  M  before  step  t. 


*M?es 

M^res 

^res 
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VL  M  =  (M,  P)  is  a  fully  specified  OCRAM  with  input  /,  x  define 

=  {y\  M  queries  y  at  time  i  <t  on  input  f,x}. 

In  place  of  stating  that  M  =  (M,  P)  is  fully  specified,  usually  M  is  said  to  run 
with  processor  bound  P.  If  F(f,  f)jaibbreviates  F(/i, /m, and  P  is 
a  second  order  polynomial,  then  F(|/l,  |x|)  abbreviates  F(|/il, \fm\,  ki|»  — >  |aJn|)- 
The  type  2  analogue  of  concatenation  recursion  on  notation  is  given  by  the 
following. 

Definition  84  F  is  defined  from  G,  H,  K  by  concatenation  recursion  on  nota¬ 
tion  (crn)  if  for  all  f^x^y^ 

F(/,f,0)  =  G(/,f) 

F{S, S, so{y))  =  F(f, f, j()^bit(0, i?(/, x, y)),  provided  that  x^O 

/«:(/,  3/)). 

Definition  85  The  type  2  functional  H  is  defined  by  weak  bounded  recursion 
on  notation  WBRN  from  if 

F(/,f,0)  =  G{f,x) 

F{f,  X,  so{y))  =  Hoif,  X,  y,  F{f,  x,y)),  iin^O 
F{f,x,si(y))  =  Hi{f,x,y,F{f,x,y)) 

H{Lx,y)  =  F{f,xM) 

provided  that  F{f,x,y)  <  K{f,x,y)  holds  for  all  f,x,y. 

Definition  86  The  algebra  is  the  smallest  class  of  functionals  (of  type  1  and 
2)  containing  0,so,sij^?5BIT,|x|,#,  Ap  and  closed  under  functional  composition, 
expansion,  and  CRN.  The  algebra  A  is  the  closure  of  0,so,si,iJ,BiT,l2:l,#,  Ap 
under  functional  composition,  expansion,  CRN  and  WBRN. 

The  following  theorem  is  the  type  2  analogue  of  the  fact  that  AC®  (or  equiv¬ 
alently  lh)  is  characterized  by  the  function  algebra  Aq, 

Theorem  87  (Clote,  Kapron,  Ignjatovic  [20])  A  functional  F{f,x)  belongs  to 
Aq  if  and  only  if  it  is  computable  on  an  OCRAM  in  constant  time  with  at  most 
P(|/|,  1^1)  many  processors,  for  some  second-order  polynomial  P. 

The  type  2  analogue  of  Theorem  26  was  established  by  the  author  and  will 
appear  in  the  journal  version  of  [20].  For  notational  brevity,  the  theorem  is 
stated  for  rank  (1,1)  functionals,  though  an  appropriate  statement  holds  for 
type  2  functionals  of  any  rank. 
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Theorem  88  (P.  Clote)  Let  F  be  a  rank  (1, 1)  functional  Then  F  e  A  if  and 
only  if  there  exist  second  order  'polynomials  P,  Q  for  'which  F  is  computed  by 
an  OCRAM  running  in  polylogarithmic  time  0(|P(|/|,  |ar|)|*)  with  a  polynomial 
Q{\f\,\x\)  number  of  processors. 

In  his  attempted  proof  of  the  continuum  hypothesis,  D.  Hilbert  [48]  studied 
classes  of  higher  type  functionals  defined  by  the  operations  of  composition  and 
primitive  recursion.  Hilbert’s  general  scheme  ([48],  p.  186)  was  of  the  form 

F{G,H,0)  =  H 

P(G,  P,  n  +  1)  =  G{FiG,  P,  n),n) 

where  F^G,H  are  higher  type  functionals  of  appropriate  types  possibly  having 
other  parameters  not  indicated.  Illustrating  the  power  of  primitive  recursion 
over  higher  type  objects,  Hilbert  gave  a  simple  higher  type  primitive  recursive 
definition  of  the  Ackermann  function. 

Higher  type  functional  complexity  theory  is  an  emerging  field.  For  reasons 
of  space,  only  references  to  a  few  recent  papers  will  be  given.  In  [59]  Ker- 
I  Ko  gave  a  survey  of  results  concerning  sequential  complexity  theory  of  real 
valued  functions.  In  [49,  50],  H.J.  Hoover  investigated  parallel  computable  real 
valued  functions.  In  [28],  S.  Cook  gave  a  survey  of  higher  type  computational 
approaches,  and  proves  Theorem  80.  Cook  further  proposed  that  any  class 
C  of  feasible  type  2  functionals  must  satisfy  the  following  two  conditions:  (1) 
BFF  C  (7  C  OPT,  (2)  C  is  closed  under  abstraction  and  application.  In  [81] 
A.  Seth  defined  a  class  C2  of  type  2  functionals  defined  by  counter  Turing 
machines  with  polynomial  bounds,  which  satisfies  the  previous  conditions,  and 
proved  that  no  recursively  presentable  class  of  functionals  exists  which  contains 
C2  and  satisfies  the  previous  conditions.  In  [82]  Seth  further  investigated  closure 
conditions  for  feasible  functionals.  In  [76],  J.  Royer  studied  a  polynomial  time 
counterpart  to  the  Kreisel-Lacombe-Shoenfield  theorem  [60]. 

Complexity  theory  for  functionals  of  all  finite  types  was  initiated  by  S.  Buss, 
who  in  [12]  introduced  a  polynomial  time  analogue  of  the  hereditarily  recur¬ 
sive  operations  HRO  to  define  polynomial  time  functionals  of  all  finite  types 
decorated  with  runtime  bounds.  A.  Nerode,  J.  Remmel  and  A.  Scedrov  [67] 
studied  a  polynomially  graded  type  system.  In  [34],  J.-Y.  Girard,  A.  Scedrov 
and  P.  Scott  introduced  bounded  linear  logiCy  and  prove  a  normalization  the¬ 
orem  which  yields  a  characterization  of  a  feasible  class  of  type  2  functionals. 
In  [30]  S.  Cook  and  A.  Urquhart  introduced  an  analogue  of  Godel’s  system  T 
by  admitting  a  recursor  for  bounded  recursion  on  notation  for  type  1  objects. 
Their  system  PV^  provided  a  natural  class  of  polynomial  time  higher  type  func¬ 
tionals  (called  the  basic  feasible  functionals  of  higher  type).  In  [45],  V.  Harnik 
extended  Cook-Urquhart’s  functionals  to  levels  of  the  polynomial  time  hierar¬ 
chy.  In  [29]  S.  Cook  and  B.  Kapron  characterized  the  higher  type  functionals 
in  PV^  by  certain  kinds  of  programming  language  constructs,  typed  while  pro¬ 
grams  and  bounded  loop  programs.  This  kind  of  characterization  was  extended 
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by  P.  Clote,  B.  Kapron  and  A.  Ignjatovic  in  [20]  to  the  higher  type  functionals 
in  NC^,  relating  hounded  loop  programs  with  higher  type  parallel  complexity 
classes.  In  [83]  A.  Seth  extended  his  definition  of  counter  Turing  machine  to 
all  finite  types,  thus  characterizing  PV^  by  a  machine  model.  If  one  addition¬ 
ally  allows  dynamic  computation  of  indices  of  subprograms  within  this  counter 
Turing  machine  model,  then  Seth  has  conjectured  this  class  to  properly  contain 

pyo; 

In  [63]  D.  Leivant  and  J.-Y.  Marion  gave  various  characterizations  of  ptime 
by  typed  A-calculi  with  pairing  over  an  algebra  W  of  words  over  {0, 1}.  In 
an  unpublished  paper,  the  same  authors  showed  how  a  natural  restriction  of 
functional  recurrence  with  substitution  generates  exactly  PSPACE.  In  a  series 
of  papers  (see  for  instance  [62])  D.  Leivant  investigated  various  tiering  schemes 
of  recursion  (extensions  of  safe  recursion)  and  related  complexity  classes.  Such 
investigations  may  have  some  applicability  to  programming  language  design.  In 
[68],  building  on  work  of  H,  Schwichtenberg  [80],  K,-H.  Niggl  investigated  certain 
subrecursive  hierarchies  (analogues  of  primitive  recursive)  of  partial  continuous 
functionals  on  Scott  domains.  Higher  type  functional  complexity  is  currently 
an  active  field  and  likely  to  remain  so  for  some  time. 
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Abstract 

It  is  notoriously  hard  to  express  computational  complexity  properties 
of  programs  in  programming  logics  based  on  a  semantics  which  respects 
extensional  function  equality.  This  is  a  serious  impediment  to  certain 
key  applications  of  programming  logics,  even  those  which  apply  very  well 
otherwise. 

This  paper  shows  how  to  define  computational  complexity  measures  in 
such  logics  as  long  as  they  support  inductively  defined  types,  dependent 
products,  and  functions.  The  method  exploits  a  natural  feature  of  induc¬ 
tive  definitions  in  type  theory,  namely  that  implicit  codes  are  kept  with 
the  objects  showing  how  they  are  presented  in  the  inductive  class. 

The  adequacy  of  the  proposed  definition  depends  on  a  faithfulness  the¬ 
orem  showing  that  the  external  (or  meta-level)  definition  of  complexity 
is  respected  by  the  internal  definition.  The  results  are  applied  to  defin¬ 
ing  resource  bounded  quantifiers  that  can  be  used  to  state  complexity 
constraints  on  constructive  proofs  and  their  extracted  programs.  In  such 
resource  bounded  logics  it  is  possible  to  prove  theorems  like  a  PTime  ax¬ 
iom  of  choice.  The  results  of  the  paper  bridge  the  fields  of  semantics  and 
complexity  to  a  small  extent. 


1  Introduction 

Most  programming  logics  use  this  rule  for  function  equality 
/  =A^B  9  iff  Vx  :  A.  f{x)  =b  9{x). 

The  functions  /  and  g  may  be  given  by  programs,  say  that  /  and  g  are  also 
names  for  the  programs.  In  the  meta-theory  of  the  programming  logic,  we  have 
access  to  a  finer  equality,  the  equality  on  /  and  g  as  programs  or  terms.  Given 

*  Work  supported  in  part  by  NSF  grant  CCR-9244739  and  ONR  grant  N00014-92-J-1764 
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this  access  to  the  program  structure,  we  can  define  the  usual  computational 
complexity  measures,  say  time(^f)  and  spctce(/).  But  in  the  object  theory,  we 
lose  access  to  these  functions  since  they  do  not  respect  the  function  equality. 

One  approach  to  gain  access  to  computational  complexity  in  the  object  logic  is 
to  define  a  finer  equality  on  functions,  say  some  intensional  equality  [10].  But 
experience  has  shown  that  such  logics  are  difficult  to  use  and  to  interface  with 
conventional  mathematics.  Another  approach  is  to  use  a  logic,  say  Bounded 
Linear  Logic  [16],  that  keeps  track  of  computational  resources.  This  approach 
also  requires  a  great  deal  of  as  yet  unfinished  work  to  show  that  such  an  axiom- 
atization  of  programming  is  manageable. 

The  issue  in  this  paper  is  to  look  for  an  existing  mechanism  in  a  constructive 
programming  logic  that  can  be  exploited  to  define  computational  complexity  in 
the  object  logic  in  a  natural  way. 

The  basic  idea  is  to  notice  that  the  computational  interpretation  of  an  inductively 
defined  class  of  functions,  say  C{A  B)  defined  over  A  B,  contains  in  it  an 
implicit  system  of  codes  that  can  be  used  to  define  complexity,  ^^e  will  show 
that  given  /  G  C(A  B)  we  can  define  time(f)  and  space(f)  in  terms  of  these 
codes.  This  is  possible  because  equality  bn  inductive  classes  is  not  the  same  as 
equality  in  the  underlying  type.  We  are  exploiting  a  fact  that  is  naturally  part 
of  constructive  inductive  definitions,  not  introducing  a  new  mechanism  just  for 
the  sake  of  defining  complexity.  This  mechanism  is  also  present  in  programming 
languages  like  ML  that  support  inductive  types. 


2  Basic  Concepts 

The  results  are  given  for  an  especially  simple  but  powerful  type  theory.  Its  base 
types  are  just  unit,  1,  and  void  (true  and  false  under  propositions-as-types)  and 
the  types  Typti.  The  element  of  unit  is  just  a  dot,  •.  The  constructors  are 
dependent  functions  (11)  and  dependent  products  (S).  We  denote  the  language 
by  n/iS+  (a  Greek  acronym  for  “Promise”). 

Bx  :  A.  B{x)  are  those  functions  A(x.  6)  such  that  for  all 

a  6  A,  /(a)  G  B(a)  (/(a)  is  also  written  ap(/,  a)). 

Ex  :  A.  B(x)  are  those  pairs  <  a,  b  >  such  that 
a  G  A  and  b  G  B(a). 

It  mugt  be  that  A  is  a  type  and  B(a)  is  a  type  for  each  a  G  A.  We  also  allow 
the  disjoint  union,  A  +  B,  of  types  A  and  B.  The  elements  are  the  injections, 
inl(a)  and  inr(b). 

Finally,  //(X.  B)  is  a  recursive  type,  in  Typa ,  provided  B  is  a  monotone  function 
Typci  to  Typei,  We  say  that  a  G  ^(X.  B)  iff  a  G  F[p(X.  F)/X].  With  each 
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recursive  type  //(X.  F)  there  is  a  recursion  combinator,  fir^nd,  which  defines  the 
fi{X.  F)  -recursive  functions.  The  typing  rule  is 

Hha£fi(X.F)  X  iTypej,  z:F,  f:X  g  e  G 

H  h  fi~4nd{a;  z,  f.  g)  EG 


examples 

example  1:  The  natural  numbers  can  be  defined  as  ^{N.  1-f  AT)  with  0  =  m/(-) 
and  s«cc(ic)  =  inr{x).  The  primitive  recursive  functions  are  defined  by  the 
combinator  fi-4nd  (details  are  not  critical  to  the  results). 

example  2:  We  define  the  set  of  functions  N  ^  N  built  from  given  base 
functions  B  by  composition.  Suppose  the  composition  operator  is  C(/,  g)  = 
A(x.  f{g{x))).  Intuitively  we  are  defining  C{N  N)  —  B  Comp{C{N  — >■ 
X),  C{N  N))  where 


Comp{Si,  S2)  =  {f  •  N  N  \  3gi  :  3g2  :  6^2*  /  =  C{fnn{gi)Jun{g2))}- 

The  official  definition  of  C{N  N)  is  p{F,  B  +  Comp{F,  F)).  Notice  that  if  bi 
are  base  functions,  then  an  object  like  inr{C{inr{C{inl{bi),  inl{b2)))^  inl{bi))  is 
an  element  of  e{N  N).  Given  /  E  C{N  N),  we  need  the  mapping 

fun  :  C{N  N)  ^  (N N) 
which  “pulls  out”  the  function  part  of  the  object,  e.g. 


/un(mr(C(mr(C'(in/(6i),  m/(62))),  tn/(6i)))  =  C{C{bi,  62),  61) 

There  is  another  part  of  the  object  hidden  in  the  definition,  namely  the  se¬ 
quence  of  injections  ini,  inr  and  the  access  to  B.  So  we  can  define  code{f)  = 
2nr(mr(m/(l),in/(2)),  m/(l)).  This  code  tells  us  the  structure  of  the  definition 
of  fun{f).  Based  on  the  code  we  can  measure  the  complexity  of  this  particular 
presentation  of  /  based  on  using  a  specific  means  of  computing  a  function  in 
N  N  that  is  guaranteed  to  be  equal  to  /. 


3  Defining  Computational  Complexity 

3.1  Meta-level  complexity 

A  careful  definition  of  the  language,  11/2^+  (Promise),  would  be  based  on  a  class 
of  terms.  The  definition  would  include  these  clauses,  (It  can  define  most  of 
Nuprl  [8].) 


t  E  term 
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X  E  var 
X  E  term 


inl{t)  E  term 
inr(t)  E  term 


X  E  var,  t  E  term 
A(x.  <)  E  term 


a  E  term  f  E  term 
^P{fi  «)  ^  term 


a  E  term,  6  E  term  x  e  var  t  e  term  t  E  term,  z  E  var  /  E  ierm  E  term 
pair{a,  b)  E  term  fji(x.  t)  E  term  fi-ind{t;  z,  f.  g)  E  <erm 

Computation  is  defined  by  structured  operational  semantics,  for  example,  we 
have  these  rules  among  others. 

flX{x,b)  b[a/x]lc  g[a/z,  X(x.  fjL-ind{x;z,  f.g)/f]  j  c 
«)  i  c  p-ind{a;  z,  _f.  g)  [c 

We  can  define  computational  complexity  based  on  this  kind  of  scheme  or  on  a 
rewrite  sematnics.  For  example,  a  simple  step  count  can  be  defined  by 

time{f)  I  n  time{b[a / x])  [  m 
time{ap{f,  a))  |  (n  +  m  +  1). 

This  approach  allows  us  to  define  both  an  evaluation  function  and  a  time- 
complexity  function  on  terms.  Given  /  a  function  term 

eval{f)  :  {x  :  term  \  3y  :  termf{x)  i  2/}  term 
time{f)  :  {x  :  term  \  3y  :  term.  f(x)  J.  j/}  — ►  H. 

These  are  the  metalevel  evaluation  and  time-complexity  functions.  In  some  sense 
they  give  the  actual  complexity  of  terms  relative  to  a  particular  implementation 
of  the  programming  language. 


3.2  Reflected  computation 

In  a  language  with  recursive  types  and  dependent  types  like  n/iS+  it  is  easy 
to  reflect  the  term  structure  and  evaluation  structure  into  the  object  language, 
building  the  internal  type  Term  and  the  internal  Eval  and  Time  functions.  This 
was  done  in  detail  for  Nuprl  [1].  We  will  briefly  refer  to  these  ideas  later. 
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3.3  Computational  complexity 

summary  of  problem  and  a  solution 

The  technical  barrier  between  semantic  theories  and  complexity  theory  is  that 
semantics  deals  with  (computable)  functions  in  order  to  interface  with  conven¬ 
tional  mathematics,  and  complexity  theory  deals  with  algorithms  since  costs 
depend  on  the  details  of  the  algorithm.  This  distinction  comes  down  mainly  to 
this  rule  for  function  equality  used  in  most  programming  logics. 

/  =A^B  9  iff  Va;  :  A.  f{x)  =b  9(x). 
inductive  classes  of  functions 

Let^s  look  at  the  form  of  inductive  definitions  in  type  theory  [8,  9,  26,  14].  The 
ideas  are  similar  to  those  in  Feferman  and  the  methods  are  like  those  of  Scott 
[27].  If  jP  is  a  monotone  operation  on  types,  say  X  C  Y  =>  F{X)  C  F{Y), 
then  fjL{X.  F)  is  a  type  that  is  essentially  the  least  fixed  point  of  F.  This  fact 
is  characterized  by  equipping  ii{X,F)  with  an  induction  principle.  Informal 

notations  often  look  like  X  F{X).  So  we  might  write  N  1  A  N  ioi 
-H  iV). 

We  are  going  to  define  a  class  of  functions  that  has  the  form  pt{F.  Base-f-Rec(F)-|- 
Comp(F))  where  Base  is  a  collection  of  base  functions  and  Rec  and  Comp  define 
classes  based  on  a  recursion  combinator  and  composition.  We  will  pick  Base, 
Rec,  and  Comp  so  that  the  classes  defined  are  the  polynomial  time  functions. 
The  key  to  doing  this  elegantly  is  in  the  work  of  Leivant  [22]  from  LICS  ’91;  the 
particular  result  I  use  (at  Leivant ’s  suggestion)  is  from  Bellantoni  and  Cook  [3]. 

Let  N  =  {0,1}  list.  This  represents  the  natural  numbers  in  the  usual  way  (with 
degenerate  leading  0,  low  order  bits  at  the  head).  Let  =  1,  the  unit  type, 
and  =  AT  X  iV”. 

The  functions  we  study  have  two  kinds  of  numerical  inputs,  called  normal  and 
safe  in  [3].  We  think  of  them  zis  canonical  inputs  (or  normal)  and  non-canonical 
(or  non-normal).  So  they  have  type  TV”  x  N.  The  left  arguments  are 

canonical  (N^).  We  use  to  indicate  the  absence  of  an  input.  The  various 
function  spaces  are  collected  into  a  single  domain,  HD,  by  taking  the  disjoint 
union. 

Def.  ©  =  ‘  N. 

The  form  of  any  element  of  ©  is  {n,m,f)  for  /  the  function.  The  selection 
functions  for  /  G  ©  is  norm{f)  =  n,  safe{f)  =  m,  fun{f)  =  /. 

Let  {x  :  II  Bs)  abbreviate  the  dependent  product  type,  Ex  :  A.  Bx.  The 

elements  are  (a,  h)  with  a  £  A  and  h  £  Ba.  We  also  use  the  Nuprl  phrase 
I  (P(ic))  for  P  a  proposition.  This  is  a  proposition  whose  computational  content 
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has  been  “squashed.”  The  official  definition  is  {unit  |  P(x)},  so  the  value  is  •  if 
P(x)  is  true,  otherwise  the  type  is  empty. 

The  Base  functions  are  divided  into  five  classes  Co, . . . ,  C4. 

Def. 


Co  =  {f  :  D\  norm{f)  =  0 
&  safe{f)  =  0 
&  fun{f)  =  Ap.  0} 

Cl  =  [f  :  D\  3n,  m,  i :  N.  norm{f)  =  n 
k  safe{f)  -  m 
k  fun{f)  =  proj(n,  m,  e) 

&  if  n  =  0  then 

if  m  =  0  then  i^n  +  X] 

These  are  the  projection  functions  where  proj{i,  n,  m)(2?i,  a?n+i,  • .  • ,  a^n+m) 

=  Xi  provided  we  don’t  project  out  the  element  of  1. 

C2  =  {/  :  B  \3j  :  [0, 1].  norm{f)  =  0  k  safe{f)  =  1  k  fun{f)  =  Sj}.  These  are 
the  two  successor  functions,  so(-;  a?)  =  Ox  and  si(*;  x)  =  lx. 

C3  =  {/  :  B  I  norm{f)  =  0  k  safe{f)  z=  I  k  fun{f)  =  pred}.  This  is  the 
predecessor  function,  pred(-;0)  =  0  and  pred{-]  ix)  =  x. 

C4  =  {/  :  B  I  norm{f)  =  0k  safe{f)  =  3  &  fun{f)  =  cond}. 

This  is  the  conditional  function  where 

cond{-;  a,  6,  c)  =  if  a  mod  2  =  0  then  b  else  c  fi. 

Notice  that  we  separate  the  canonical  arguments  from  the  non-canonical  with  a 
semicolon. 

Along  with  these  types,  we  introduce  constructors  to  build  elements:  baseO 
build  the  zero  function,  (0,  0,  Ax.O),  6ase2(n,  m,  i)  builds  the  projection,  base2[i) 
builds  the  successor  and  baseS  the  predecessor  and  6ase4  the  conditional. 

Recursion  is  allowed  “on  notation,”  that  is,  we  can  define  a  function  f  by  recur¬ 
sion  as 


/(0,x;a)  =  g{x]a) 

f{iy,x;a)  =  hi(y,x\  f{y,x;a),a). 

We  introduce  the  recursion  combinaior  Rec{n,  m)  to  accomplish  this  form  of 
recursion.  Its  type  is 

(AT”  X N)  X N)  x  N)  ( AT’^+^x 


The  combinator  reduces  as  follows: 


Rec{n,m){g){hi){h2){0,x;a)  =  g{x;a) 

Rec{n,rn){g){hi)(h2){iy,x]a)  =  hi(y,x\Rec{n,rn){g){hi){h2)(y,x]a),a). 

We  also  use  a  “safe  composition”  combinator  CoTnp{n,  m)  whose  type  is, 

(N^xN^^N)-^  {N^  xN^^  JVf  (iV”  x  xN^^N), 

Given  h  e  x  N ,  r  e  {N^  x  ^  iV)”  and  t  E  {N^  x  N)^, 

then 

Comp(n,  m)(/i)(r)(t)(^;  d)  =  h{r{x;  •);  i{x;  a)) 

This  form  of  composition  can  be  used  to  write  in  combinator  form  a  function 
expression,  say  f{x;a),  in  terms  of  safe  composition  and  projections  as  long 
as  there  is  no  subexpression  g{e\\e2)  with  a,-  appearing  among  the  canonical 
arguments,  ei. 

Using  these  constructs  we  are  almost  ready  to  define  a  class  B  which  will  be 
PTime. 

First  let  Rec(5)  =  {/  :  B  ||  3n,  m  :  iV.  3flf,  hi ,  /12  :  |  {norm{g)  =  n 

k  safe{g)  =  m 
k  norm(hi)  =  n  +  1 
k  safe{hi)  =  m  +  1  for  i  =  1, 1 
k  fun{f)  =  Rec{n,m){fun{g)){fun{hi)){furi{h2)))} 

Comp(JB)  =  {/-^^ll  3n,m  :N3h  .  B,3f  :  B^.3t  :  B^,  l{norm{h)  =  n 

k  sa/e(h)  =  m 
&  Vi  :  [1,  n].  {norm{ri)  =  n 
k  safe{ri)  =  0) 

k  Vj  :  [1,  m].  {norm{ij)  =  n  k  safe{tj)  =  m) 
kfun{f)  =  comp{n,m){fun{h)){fun{r)){fun{i)))}) 

The  functions  fun()  used  in  these  definitions  are  defined  to  produce  the  function 
part  of  elements  of  the  recursive  type.  This  is  not  exactly  the  same  eis  for  the 
elements  of  Base  since  we  must  account  now  for  the  position  of  Base,  Rec(B) 
and  Comp(B)  in  the  disjoint  union,  but  it  is  a  polymorphic  function  defined 
independently  of  the  recursive  type. 

Among  the  natural  constructors  to  associate  with  B  are  the  ones  for  Base,  now 
extended  to  B,  i.e.  baseO  is  modified  to  wrap  “ini”  around  (0(0,Ap.0))  to  give 
m/((0,  {0,  Ap.O))),  We  also  want  rec{n,m){g){hi){h2)  and  comp{n,m){h){r){t) 
where  gf,  hi,  /12,  h  are  in  B  and  r,?  are  vectors  of  elements  of  B. 

Def.  B  =  fi{B.  Base  +  Rec(B)  +  Comp(S)) 
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Theorem  (Bellantoni  Sc  Cook): 

•  for  every  /  €  PTime,  there  is  an  f  £  B  such  that  f{x)  z= 

•  for  every  f  e  B,  f(x,y)  is  in  PTime. 

In  proving  the  Bellantoni  and  Cook  theorem  we  need  to  define  a  complexity 
me8Lsure  on  elements  of  B,  This  is  easy  because  the  constructive  treatment  of 
inductive  types  provides  with  each  element  of  the  type,  say  B,  a  code  showing 
how  it  is  built.  For  example,  a  constructor  like 

rec(0,  l){bas€  1(0, 1,2))  {comp){0,  l)(6ase2(0))(-)(6ase  1(0, 2, 3)) 

{comp{0,  l)(6ase2(l))(-)(6ase  1(0, 2, 3)) 

not  only  builds  a  function  in  B,  but  it  codes  up  a  complete  description  of  how 
the  element  is  constructed. 

The  equality  relation  on  element  of  B  is  naturally  defined  to  respect  the  coding 
of  elements,  so  it  is  not  extensional  function  equality.  This  has  always  been  a 
feature  of  inductive  definitions,  and  we  now  intend  to  exploit  it. 

So  with  each  /  €  B  we  can  define  not  only  fun{f)  but  a  function  code{f)  which 
is  the  construction  history  of  /.  We  can  think  of  these  codes  as  an  implicit  pro¬ 
gramming  language  that  comes  with  every  inductive  definition.  It  is  essentially 
just  the  expression  language  of  the  constructors  internalized  in  some  natural 
way. 

other  inductive  classes 

The  same  technique  used  to  define  B  can  clearly  be  used  to  define  the  elementary 
functions,  B,  or  the  primitive  recursive  ones,  Prim.  For  any  of  these  classes,  the 
codes  provide  a  way  to  define  resource  bounds  such  as  iime{f)  or  space{f). 

//-recursive  functions 

It  is  interesting  that  we  can  use  another  natural  feature  of  the  Nuprl  type  theory 
to  define  an  internal  model  of  the  general  recursive  functions.  (The  interpreta¬ 
tion  of  this  result  is  bound  to  be  provocative.) 

Consider  the  set  of  functions  F  =  En  :  N)  and  the  subset  F 

{/  :  F  I  :  iNT.  arity{f)  =  n  ISc  "ix  :  .  3y  :  N.f(x,  y)  =  0}. 

On  the  subset  we  can  define  fjLy.{f{x,  t/)  =  0)  as  the  least  y  such  that  f{x,  y)  =  0. 
In  Nuprl  we  can  just  use  mu  from  section  2.2. 

Following  Kleene  [21]  we  can  define  the  general  recursive  functions  over  F,  B(F), 
using  this  clause 

Mu(C')  =  {f  :  W  \  3n  :  N  3g  :  C.  arity{f)  =  n  -f  1 
Sc  i  (Va: :  N^.  3y  :  N.  fun{g)(x,  y)  =  0) 
k  fun{{f)  =  Xx.ny{g{x,  y)  =  0)} 

ii(IF)  =  fi{F.  Base  +  Prim  Rec(F)  +  Comp(F)  +  Mu(f )). 
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This  definition  provides  an  implicit  programming  language  for  the  general  recur¬ 
sive  functions.  The  operation  J,  (P)  is  a  hiding  operation  which  hides  the  proof 
that  a  y  exists. 

3.4  Computational  complexity  measures 

Given  a  set  of  codes,  Code(C),  for  an  inductive  definition,  we  can  assign  a  mea¬ 
sure  of  resource  expenditure.  In  general,  this  can  be  any  computable  function 
which  respects  the  arity  of  the  specified  function,  that  is  arity  satisfies 

fun{f)  :  _  B 


and  a  measure  satisfies 


M[code{f))  :  -  N. 

We  are  only  interested  in  measures  which  reflect  complexity  measures  that  can 
be  imposed  on  the  term  structure.  (Here  we  could  require  that  they  be  Blum 
measures  for  example  [4].)  We  want  the  following  faithfulness  condition  for  a 
measure. 

Definition:  A  measure  of  computational  complexity  M  on  a  class  C{A*  — »•  B)  is 
faithful  iff  there  is  a  complexity  measure  m  on  term  such  that  for  all  /  €  C{A*  — ► 
P),  there  is  a  term  g  such  that  fun{f)  =a*-^b  9  M (code{f)){x)  =  m(p)(»). 

This  condition  says  that  M  is  actually  a  measure,  a  complexity  property  that 
is  definable  on  the  terms  using  the  evaluation  relation  of  the  implementation  of 
the  programming  language  and  its  logic. 

Definition;  We  define  the  Time  measure  on  the  primitive  recursive  funtions, 
over  a  free  algebra  A  with  constructors  c,*,  say  PR(A*  — ►  A),  in  the  usual  manner 
[23]. 

Let 

Up  (a?  1 ,  .  .  • ,  Xn)  =  Xi 

Time{Up)  =  1 

Ti7ne{h{gi{x),  . . p„(®)))  = 
n 

^Time(pj)(i)  +  Time{h){gi(x),  . . . , 

t=i 

Time{Rg){ci{d),  x)  = 

Time{gci){Rg{ai,  x),  ...,  Rg{any  x),  d,  «)+ 
n 

yyrime(Rg){ai ,  «) 

»=i 

where  Rg  is  a  brief  notation  for  the  primitive  recursive  function  definition. 
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Theorem:  Time  is  a  faithful  complexity  measure  on  n/ill+. 

The  proof  is  just  a  matter  of  showing  that  Time  mimics  the  time  measure  on 
terms.  This  can  be  done  because  we  have  picked  a  class  that  uses  only  first  order 
functions.  It  is  more  interesting  to  show  that  this  can  be  done  for  inductive 
classes  that  use  type  2  functions  and  higher.  The  result  for  type  2  can  be  proved 
using  Melhorn’s  definitions  [25,  7]  or  those  in  Bellantoni  Cook  [3]. 


4  Resource  Bounded  Logics 

4.1  Coding  logic 

As  is  well  known,  the  predicate  calculus  can  be  represented  in  a  type  theory  such 
as  n//E+  using  the  propositions-as-types  principle.  The  key  points  are  that  the 
universal  quantifier  "ix  :  A.  B  is  represented  as  a  function  space,  Ux  :  A.  B  and 
the  existential  quantifier  3x  :  A,  B  is  represented  as'Ex  :  A,  B. 

The  representation  of  logic  allows  us  to  state  program  specifications  as  proposi¬ 
tions  to  be  constructively  proved.  For  example,  the  problem  of  finding  the  least 
prime  factor  of  a  number  can  be  stated  using  {2  •  •}  =  {a?  :  TV  |  2  <  a?}  : 


Vn  :  {2  •  :  TV.  p  is  the  least  prime  factor  of  n 


We  would  like  to  be  able  to  put  a  complexity  constraint  on  this  specification, 
asking  that  p  be  found  in  polynomial  time  say. 

4.2  Resource  bounded  logics 

We  want  to  define  the  class  of  feasible  proofs  in  type  theory.  The  major  criterion 
is  that  for  the  types  needed  in  number  theory  (specifically  HA^^),  say  A  and  B, 
if  we  feasibly  prove  Va?  :  A.  3y  :  B.  R(x,y),  then  3f  :  Poly{A  — >■  5).Vai  : 
A,R{x,  f(x))  where  Poly{A  — >•  B)  are  the  polynomial  time  functions  from  A  to 
S.  The  definition  of  Poly(A  — »•  B)  follows  the  approach  to  the  complexity  of 
higher-order  operators  taken  in  Constable[7],  Mehlhorn  [25],  Cook  k  Urquant 
[11], 

In  order  to  define  feasible  proofs,  we  use  the  propositions-as-types  principle  and 
define^^Va:  :  A.3yB.R{x,y)  as  the  function  type  Jl{A;x.'L{B.y.R{x,y)))  (or  in 
Nuprl  notation,  x  :  A  y  :  B  x  R(x,y)).  We  need  to  define  a  notion  of 
polynomial  time  function  in  the  dependent  function  class  n(A;  x.B).  We  follow 
the  method  of  defining  complexity  bounds  over  elements  of  an  inductively  defined 
class  large  enough  to  include  the  objects  we  want. 
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numerical  types 

We  first  define  the  types  needed  for  number  theory  (HA^). 

T  =  ^(T.  {A  :  (7i  II  =  N  V  A  =  void  V  3  n,  m  :  N.A  =  Eq{n,  m)}  + 

{A  :Ui\\3BuB2:T,l(A  =  ty{Bi)  x  ty{B2))}  + 

{A  :Ui\\3BuB2:T.liA  =  iy{Bi)  +  fy(B2))} 

{A  :Ui\\3BuB2:T.l{A  =  ty{B,)  ty{B2))}  + 

{A:Ui\\3B:T.3P:  ty{B)  Ui.  I  {A  =  i:x  :  ty{B).  P{x))} 
{A:Ui\\3B:T,3P:  ty{B)  -^Ui.l{A  =  nx:  ty{ByP{x))}) 

Notice  that 

{A:Ui\3T:T.A  =  ty{T)}CUi. 

Let  T  =  ST  :  T.  T;  the  elements  are  pairs  {T,  t)  where  T  G  T  and  t  E  T. 
Our  goal  is  to  define  the  subclass  of  “polynomial  time”  elements  of  called 
Poly(P).  We  will  define  this  as  a  subtype  of  inductively  defined  class  called 
Tl{P),  using  the  measure  of  computational  complexity  for  higher  type  objects 
mentioned  above  [7,  25,  11]. 

We  will  define  to  be  the  least  class  containing  elements  of  N  and  “elements” 
(proofs)  of  the  atomic  types,  the  successor  functions,  the  identity  functions,  and 
closed  under  pairing,  selection,  composition,  constants,  explicit  operations  on 
arguments  (permutation  of  arguments,  duplication  of  arguments  and  applica¬ 
tion)  and  primitive  recursion.  By  the  results  of  Grzegorczyk  [17]  this  class  will 
include  all  the  primitive  recursive  objects  of  higher  type.  Then  we  define  com¬ 
plexity  functions  and  restrict  72.(T')  to  the  polynomial  time  computable  objects 
of  finite  type,  Poly(P).  The  details  follow  the  pattern  of  3.3  and  are  omitted 
here.  Basically  is 

li{F,  Base  -}-  Constants{F)  +  Pairs{F)  -f  Explicit{F)  -|-  Recur sion{F)). 

Once  we  can  define  Poly{T)  it  is  possible  to  express  the  condition  we  needed 
above  using  propositions-as- types.  For  any  type  T  G  T,  let  Poly{T)  be  those 
functions  of  Poly{T)  of  type  T.  Then 

Mpoiyx  :  N.  P{x)  ==  Poly{Ux  :  N.  P{x)). 

In  general  for  a  complexity  class  C^{Jlx  :  A.  P{x))  we  can  define 


Vera? :  A.  P(x)  =  C'^iUx  :  A.  P(x)). 
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Abstract 


The  abuse  of  structural  rules  may  have  damaging  complexity  eifects. 


1  INTRODUCTION  :  A  LOGIC  OF  POLYTIME  ? 

We  are  seeking  a  «  logic  of  poly  time  ».  Not  yet  one  more  axiomatization,  but 
an  intrinsically  poly  time  system.  Our  methodological  bias  will  be  to  consider 
that  the  expressive  power  of  a  system  is  the  complexity  of  its  cut-elimination 
procedure,  and  we  therefore  seek  a  system  with  a  poly  time  complexity  for  cut- 
elimination  (to  be  precise  :  besides  the  size  of  the  proof,  there  will  be  an  auxiliary 
parameter,  the  depth,  controlling  the  degree  of  the  polynomial).  This  cannot 
be  achieved  within  classical  or  intuitionistic  logics,  because  of  structural  rules, 
especially  contraction  :  this  is  why  the  complexity  of  cut-elimination  in  all  extant 
logical  systems  (including  the  standard  version  of  linear  logic  which  controls 
structural  rules  without  forbidding  them)  is  catastrophic,  elementary  (towers  of 
exponentials)  or  worse.  Light  Linear  Logic  is  a  purely  logical  system  with  a  more 
careful  handling  of  structural  rules  :  this  system  is  strong  enough  to  represent 
all  polytime  functions,  but  cut-elimination  is  (locally)  polytime.  With  LLL  our 
control  over  the  complexity  of  cut-elimination  improves  a  lot. 

But  this  is  not  the  only  potentiality  of  LLL  :  why  not  transforming  it  into  a 
systems  of  mathematics,  and  try  to  formalize  «  polytime  mathematics  »  in  the 
same  way  as  Heyting  arithmetic  formalizes  constructive  mathematics  ?  The 
possibility  is  clearly  open,  since  LLL  admits  extensions  into  a  naive  set-theory, 
with  full  comprehension,  still  with  a  poly  time  cut-elimination.  This  system 
admits  full  induction  on  data  types,  which  shows  that,  within  LLL,  induction 
is  compatible  with  low  complexity. . . 
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1.1  Background 

1.1.1  Complexity  of  normalization 

Our  goal  is  to  find  a  logical  system  in  which  the  I/O  dependencies  are  given  by 
polytime  functions.  We  shall  try  a  proof-theoretic  approach,  namely  to  make 
sure  that  cut-elimination  is  poly  time.  In  fact  we  shall  concentrate  on  the  follow¬ 
ing  question  :  which  logical  system(s)  induce  normalizations  of  a  given  complex¬ 
ity  (polytime  or  not)  ? 

There  is  an  answer,  namely  MALLq  (multiplicative- additive- quantifiers  (of  any 
order)  linear  logic)  :  the  small  normalization  theorem  of  [1]  aissigns  a  size  bound 
to  proofs  ;  this  size  shrinks  during  lazy  cut-elimination,  hence  induces  linear 
time  functions.  A  crucial  technological  point  is  that  the  notion  of  cut-degree 
disappears,  i.e.  the  procedure  is  not  dependent  of  the  fact  that  cuts  are  replaced 
with  simpler  ones.  The  operation  has  perfectly  succeeded,  but  the  patient  died 
(of  starvation)  :  this  system  is  desperately  inexpressive. 

Linear  logic  wisely  has  another  stock  of  connectives,  namely  exponentials  which 
should  compensate  for  this  limitation,  by  restoring  the  necessary  amount  of 
structural  manipulations  (mainly  the  contraction  rule).  Now  the  patient  is  still 
dying,  but  of  overfeeding  :  the  complexity  is  no  longer  bounded  by  any  reason¬ 
able  measure,  since  usual  logic  (classical  or  intuit ionistic)  imbeds.  The  question 
is  therefore  to  find  more  reasonable  connectives,  sitting  in  between  MALLq  and 
LL.  These  connectives  are  the  light  exponentials. 

The  first  attempt  dates  back  from  1987  (joint  work  with  A.  Scedrov  &  P.  Scott, 
[3])  and  is  based  on  the  idea  of  replacing  \A  (which  usually  means  A  ad  libitum) 
by  (1  &  A)  0  ...  (g)  (1  &  A),  i.e.  essentially  by  a  finite  tensor  power  of  A,  !nA. 
It  is  also  immediate  to  see  that  the  rules  of  weakening,  dereliction,  contraction 
and  promotion  are  still  valid  w.r.t.  bounded  exponentials  :  the  bounds  are  re¬ 
spectively  given  by  0, 1,  +, .,  i.e.  the  maintenance  is  polynomial.  This  very  good 
starting  point  leads  to  BLL  (Bounded  Linear  Logic)  ;  BLL  has  a  lot  of  qualities 
(it  exactly  corresponds  to  polytime  etc.),  but  it  has  a  major  drawback  :  it  men¬ 
tions  the  polynomial  bounds  who  should  remain  hidden  ^ .  By  the  way  observe 
that  BLL  is  far  from  giving  good  bounds  :  the  main  property  of  exponentials 
is  the  isomorphism  between  !A0!B  and  !(A  &  5),  but  BLL  yields  the  bounds 
!„+rn(A  k  B)-o\nA0\mB  and  !„A0!n5— o!„(A  &:  B),  which  induce  by  composi¬ 
tion  bn  (A  k  B)—o\n{A  k  B)^  not  quite  an  isomorphism. 

Since  this  first  attempt,  many  other  restrictions  have  been  tried  by  Danos,  Joinet, 
Lafont,  Schellinx  and  myself,  without  obtaining  truly  convincing  results 
Other  connections  between  polytime  complexity  and  normalization  have  been 
made  in  the  recent  years,  typically  works  of  Leivant,  Leivant-Marion  [7,  8], 
and  Hillebrand-Kanellakis-Mairson,  [5].  These  approaches  stay  inside  typed  A- 
calculi,  i.e.  systems  which  are  by  no  standards  polytime  (the  complexity  is  at 
least  elementary),  but  they  individualize  certain  interesting  situations  where  the 
complexity  is  exactly  poly  time  (this  is  based  on  the  fact  that  in  traditional  sit- 


1.  These  bounds  do  not  refer  to  time,  but  to  size. 

2.  A  posteriori  it  is  clear  that  all  these  attempts  failed  because  they  included  the  principle 
[V],  see  1.1.3. 
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uations,  the  complexity  is  determined  by  the  cut-formulas  :  the  basic  idea  is 
to  restrict  one  to  cuts  of  a  certain  form  to  achieve  complexity  effects).  The 
obvious  advantage  of  these  approaches  lies  is  the  use  of  traditional  systems  (or 
at  least  systems  not  too  far  from  that).  But  these  systems  can  hardly  claim  to 
bring  some  insight  as  to  the  logical  nature  of  poly  time,  since  as  soon  as  we  iter¬ 
ate  their  logical  primitives,  the  complexity  explodes,  in  other  terms  the  logical 
primitives  (basically  intuitionistic  implication)  make  mistakes  w.r.t.  complexity. 
To  take  an  analogy  :  classical  (Peano)  arithmetic  is  indeed  constructive  for  If® 
sentences,  where  it  coincides  with  Hey  ting  arithmetic  ;  for  more  complex  for¬ 
mulas,  it  is  constructively  wrong.  There  is  therefore  still  a  want  for  a  system 
which  is  intrinsically  poly  time,  in  the  way  that  Hey  ting  arithmetic  is  intrinsically 
constructive.  An  answer  is  LLL. 

1.1.2  LLL  and  naive  set-theory 

We  are  seeking  a  logical  system  with  a  light  complexity.  This  basically  means 
that  the  cut-elimination  bounds  will  not  depend  on  the  cut-formulas,  i.e.  will 
not  rely  on  the  replacement  of  a  cut  by  a  simpler  one.  Then  such  a  system  will 
accommodate  naive  set-theory.  This  is  simply  because  naive  set-theory  has  a 
normalization  procedure  (the  one  described  by  Prawitz  in  the  60 ’s)  which  will 
terminate  in  such  a  framework.  Typically  this  works  for  MALLq  (this  is  indeed 
an  old  remark  of  Grishin,  [4]),  since  the  naive  comprehension  scheme  does  not 
prevent  normalization  from  shrinking  ! 

Our  crucial  test  for  selecting  the  right  rules,  will  be  to  check  whether  or  not 
naive  set-theory  becomes  inconsistent  with  the  proposed  set  of  rules  for  expo¬ 
nentials.  Typically,  naive  set-theory  enables  us  to  get  fixpoints  of  any  logical 
operation  (like  naive  function  theory,  i.e.  A-cal cuius),  and  it  suffices  to  check 
the  impossibility  of  getting  a  contradiction  from  fixpoint.  The  best  candidate 
is  the  one  arising  from  Russell’s  paradox,  i.e.  A  For  those  who  find  this 

methodology  surprising,  we  can  phrase  it  differently  :  inconsistency  (i.e.  failure 
of  cut-elimination)  is  the  case  of  a  cut-elimination  that  does  not  terminate,  and 
this  is  due  to  the  same  phenomenon  as  heavy  complexity,  i.e.  cut-elimination 
that  does  not  terminate  within  ’’feasible”  bounds. 

We  shall  therefore  tailor  our  light  exponentials  w.r.t.  naive  set-theory,  but  keep 
only  the  second  order  propositional  logic  arising  from  this  study.  It  would  be 
possible  to  do  much  more  :  one  can  add  the  logical  rules  of  naive  set-theory  to 
LLL,  and  this  provides  a  very  powerful  system.  In  this  system  extensionality 
fails  (as  already  observed  by  Grishin),  but  Leibniz  equality  can  do  wonders.  Inte¬ 
gers  in  unary  (or  binary)  numeration  can  be  defined,  and  full  induction  therefore 
works.  In  other  terms,  one  can  get  a  pure  logical  system  without  any  proper 
axiom,  which  contains  both  a  light  set- theory  and  a  light  arithmetic. 

1.1.3  Dissection  of  exponentials 

Exponentials  are  used  to  ’’classicize”  LL.  This  involves  a  lot  of  micro  properties, 
that  we  can  individualize  below  : 

► 


[I]  :  \{A  k  B)-o\A(S)\B  (and  !T  1) 
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►  [II]  :  \A^\B^\{AkB)  (and  1-^!T) 

►  [III]  :  from  A—oB  derive  \A-o\B 

►  [IV]  :  \A-olA 

►  [V]  :  !yl0!5— o!(A  (g)  B)  (and  !l) 

^  [Vl]:\A-^A 

►  [VII]  :  lA^WA 

The  first  two  principles  express  the  usual  isomorphism  which  is  responsible  for 
the  name  ’’exponential”  :  principle  [I]  expresses  contraction  and  principle  [II] 
expresses  weakening. 

Principle  [III]  expresses  functoriality  of  the  exponentials  and  is  absolutely  basic. 
Principle  [IV]  is  a  weak  form  of  dereliction  (i.e.  principle  [VI]). 

These  four  principles  will  constitute  the  basis  of  LLL. 

Principle  [V]  enables  one  to  give  a  multilinear  version  of  functoriality  (from 
T  t-  B  derive  \T  v~\B),  and  will  not  be  accepted  in  LLL,  although  it  is  also 
compatible  with  naive  set-theory 

In  presence  of  fixpoint  A  c:i\A-^,  it  is  possible  to  derive  the  sequent  i-  (so  no 
cut-elimination  !),  in  two  ways 

►  from  [I]  -f  [III]  -f  [VI] 

►  from  [I]  -h  [III]  +  [IV]  +  [VII] 

In  both  cases  one  first  proves  i-?A,  ?A  from  the  fixpoint  principle  i-  A,?  A  : 

►  dereliction  [VI]  yields  i-?^,  7 A 

►  [m]  yields  t-\A^77A,  then  [IV]  yields  t-7A,77A  and  [VI]  removes  the  extra 


From  1-?^,  7 A  contraction  [I]  yields  k~?j4,  and  by  fixpoint  one  gets  i-  A^  ,  which 
by  promotion  yields  in  turn  We  end  with  a  cut  between  h~7A  and 

Therefore  principles  [VI]  and  [VII]  are  definitely  excluded. 

The  failure  of  dereliction  is  the  reason  for  the  introduction  of  the  weaker  prin¬ 
ciple  [IV].  Unfortunately  it  turns  out  that  this  principle  is  too  weak  in  terms  of 
expressive  power,  and  this  is  the  reason  why  an  additional  modality  is  introduced. 

1.1.4  The  three  modalities 

In  LLL  there  are  indeed  three  modalities  !,§,?.  §  (neutral)  is  a  new  intermediate 
modality.  §  is  self-dual,  i.e.  (§yi)'^  is  and  its  intuitive  meaning  is  the 

(common)  unary  case  of  !  and  ?.  The  principles  of  LLL  are  : 

►  fflj  [II]>  [III]  (written  in  terms  of  I,?) 

3.  It  yields  another  logic  with  an  elementary  complexity  for  cut-elimination,  ELL. 
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►  from  Ay-  B  derive  §yl  i-  §5  [VIII] 

►  Ui-§A[IX] 

►  §^0§5i-§(yl(8)B)(and§l)  [X] 

[VIII]  is  just  usual  functoriality,  and  [X]  enables  one  to  get  a  n-ary  version  ; 
[IX]  is  a  compensation  for  the  want  of  dereliction. . .  observe  that  it  implies  by 
duality  §A  i-  ?A  and  is  therefore  an  improved  version  of  [IV]. 

These  principles  can  be  organized  along  a  sequent  calculus  which  enjoys  a  cut- 
elimination  with  polynomial  bounds,  as  we  shall  see  below. 


1.2  Expressive  power  of  LLL 

LLL  can  be  seen  as  a  system  of  set-theory  (or  arithmetic).  It  can  also  be  seen 
as  a  system  of  typed  A-calculus.  We  have  to  explain  how  to  encode  data  and 
poly  time  algorithms. 


1.2.1  Integers 

Remember  that  complexity  depends  on  the  representation  of  data  :  typically 
integers  can  be  given  in  tally  representation  or  in  binary  representation,  with  an 
exponential  reduction  of  their  size.  This  is  why  we  introduce  two  types,  int  and 
bint  for  integers. 

Tally  integers  can  be  given  the  type  int  =  VX.!(X  — o  X)  — o  §(X  -o  X),  where 
X  is  a  second  order  variable.  The  traditional  type  VX.!(X  — o  X)  -o  (X  — o  X) 
cannot  be  used  for  want  of  dereliction,  and  the  immediate  substitute  for  it, 
VX.!(X  -oX)— o!(X  -oX)  cannot  be  used  either,  since  the  principle 
!(A  — o  A)\\{A  —o  A)  i-!(.A  -o  A)  is  not  part  of  LLL.  Observe  that  addition 
can  be  given  the  type  int;  int  k-  int  and  multiplication  can  be  given  the  type 
int;  !int  i-  §int.  In  fact  any  polynomial  P  in  k  variables  can  be  given  ^  a  type 
int  0  ...  0  int  — o  §*int,  where  k  is  an  integer  depending  on  the  degree  of  P. 
Typically,  can  be  given  the  type  int  — o  §§int. 

Binary  integers  (lists  of  0  and  1)  can  be  given  the  type 

bint  =  VX.!(X  -oX)  0  !(X  — oX)  — o§(X  -oX).  There  is  a  canonical  map  which 
consists  in  replacing  a  binary  list  with  a  unary  one,  i.e.  ((2^)  is  the  length  of  x 
in  tally  representation.  The  type  of  this  map  is  bint  — o  int. 


1.2.2  Turing  machines 

Let  us  fix  the  alphabet  and  the  set  of  states  of  our  Turing  machines.  In  or¬ 
der  to  represent  our  machines,  all  we  now  have  to  do  is  to  find  a  type  Tur,  in 
such  a  way  that  configurations  (tape  -f  state)  of  such  a  machine  are  exactly  the 
objects  of  type  Tur.  Tur  must  also  be  such  that  the  instructions  of  a  Turing 
machine  induce  objects  of  type  Tur  -o  Tur.  Several  possibilities  are  at  hand, 


4.  Up  to  minor  details 
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but  the  simplest  is  for  sure  to  use  the  fixpoint  facility  coming  from  the  naive 
comprehension  axiom.  (The  fixpoint  of  the  operator  $[p]  is  obviously  <  G  with 
t  =  {ar  I  G  a?]})- 


1.2.3  Poly  time  functions 

Let  us  now  take  a  poly  time  program  from  binary  integers  to  binary  integers,  with 
runtime  P.  We  can  consider  the  function  of  type  bint  -o  §Tur  which  yields  the 
input  configuration  of  the  machine,  as  well  as  the  function  of  type  bint-o(§)*^inf 
which  yields  the  number  of  steps  ;  if  our  program  is  represented  by  <p  of  type 
Tur  -o  Tur,  then  we  eventually  get  an  object  of  type  bint  -o  (§)^+^Tar  which 
yields,  in  function  of  the  binary  input,  the  output  tape. 

This  representation  has  no  pretension  to  elegance  :  its  only  virtue  is  to  show 
the  expressive  power  of  LLL.  Since  LLL  is  a  real  logical  system  in  which  it  is 
impossible  to  be  worse  than  polytime,  smarter  representations  must  be  found. 


1.3  Cut-elimination 

The  sequent  calculus  naturally  associated  with  LLL  is  a  double-layer  version, 
i.e.  with  additive  and  multiplicative  disjunctions.  This  is  not  very  friendly  but 
after  all  sequent  calculus  is  not  the  only  proof-theoretic  syntax,  and  one  can  use 
more  sophisticated  technologies,  typically  proof-nets.  The  proof-net  technology 
has  made  essential  progress  in  the  recent  years,  and  it  now  possible  to  represent 
the  full  sequent  calculus,  in  terms  of  proof-nets  with  boxes.  Boxes  are  only 
needed  for  exponentials  :  the  promotion  rule  [III]  induces  a  box  as  well  as  the 
dereliction  (or  rather  its  weaker  version)  (combination  of  [ VI 11], [IX], [X]).  The 
main  parameters  of  a  proof  are 

V  its  which  is  the  nesting  number  of  the  exponential  boxes  ; 

V  its  size  which  counts  the  number  of  links  ; 

►  its  partial  sizes,  i.e.  the  size  of  the  part  of  the  net  which  is  at  a  certain  depth. 

Cut-elimination  works  as  follows  :  it  is  a  lazy  one,  i.e.  no  cut  is  eliminated 
«  inside  a  &;-box  »®,  which  is  performed  layer  after  layer,  first  starting  with 
depth  0.  After  eliminating  the  cuts  of  depth  0,  the  sizes  (which  were  so,  ,  S2,  •  •  •) 
become  at  most  sq?  sqSi ,  5052,  •  • .  From  this  it  is  immediate  that  after  eliminating 
all  cuts,  the  final  size  is  roughly  ,  where  5,  d  are  the  original  size  and  depth. 
What  makes  the  argument  work  is  that  the  light  rules  are  of  constant  depth, 
i.e.  that  no  change  (increase  or  decrease)  may  happen  during  cut-elimination. 
By  the  way,  these  bounds  are  the  simplest  refutation  for  additional  principles 

5.  The  proof  of  cut-elimination  with  the  polynomial  bounds  is  not  manageable  with  sequent 
cailculus. 

6.  Technically  this  is  the  notion  of  ready  cut  coming  from  [2]. . .  strictly  speaking  there  are 
no  longer  additive  boxes. 
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such  as  §(yl  0  B)  I-  0  :  a  function  of  type  bint  i-  §(1  0  1)  computes 

in  quadratic  time  (since  we  can  stop  the  computation  when  the  0-rule  occurs, 
whereeis  the  type  bint  i-  §1  0  §1  requires  only  linear  time. 

With  proof-nets,  it  is  easy  to  see  that  the  bound  immediately  yields  a  time 
bound  for  usual  I/O,  like  binary  strings.  Moreover  a  binary  string  is  represented 
by  a  proof-net  of  depth  1,  hence  the  application  f{s)  of  a  given  function  /  of 
type  bint— o(§)*Tur  to  a  binary  string  s  will  have  the  same  depth  as  /,  i.e.  the 
computation  will  run  in  a  time  which  is  polynomial  in  the  size  of  s. 


2  THE  SYNTAX  OF  LLL 

Constructive  logic  is  basically  propositional  ;  this  is  why  we  focus  on  (second- 
order)  propositional  LLL.  However  the  system  is  quite  flexible  and  accepts  quan¬ 
tifiers  of  any  order,  including  set-quantifiers. 


2.1  The  formulas  of  LLL 

LLL  has  the  same  connectives  as  usual  linear  logic  but  for  the  exponentials  : 
there  is  an  extra  (self-dual)  modality,  §  (neutral)  is  added 

Definition  1 

Literals  (T )  and  formulas  (F )  are  defined  as  follows 
T  =  a, /?,7. i-*-... 

F  =  T,1,X,0,T,!F,§F,?F,F®F,^’  2?  F,F  k  F,F  ®  F,^aF,3aF 

Definition  2 

(Linear)  negation  is  a  defined  connective  : 

^  (a)"*"  =  (a ■*■)■*•  =  a 

.  1-L  =  X,±-L=:1 

►  0-^  =  T,T-L  =  0 

►  (U)-L  =  =  \A^ 

^  (A  k  B)^  =  e  B^,  (A  ®  B)^  =A^kB^ 

.  (Va^)-L  =  3aX-^, (3aX)-L  =  Va^-^ 

Linear  implication  is  a  defined  connective  : 

A^B  =  A^^B 


7.  We  have  been  tempted  to  replace  !, §,  ?  by  the  musical  symbols  |j,  1], b 
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2.2  The  sequent s  of  LLL 

Definition  3 

p-  A  discharged  formula  is  an  expression  [A],  where  A  is  a  formula  ; 

p  A  block  A  is  a  sequence  Ai,. An  of  formulas,  or  a  single  discharged 
formula  [A]  ;  the  standard  case  is  that  of  a  block  of  length  1,  for  which 
we  use  the  notation  A  or  [A]  ; 

p  A  sequent  is  an  expression  i-  ; . . . ;  An,  where  A-^,. . An  are  either 
discharged  formulas  or  blocks.  The  standard  case  is  that  of  a  sequence 
of  (undischarged)  formulas  ;  even  more  standard  is  the  case  when  the 
sequence  consists  of  exactly  one  formula. 

Remark.  — 

p  A  block  Ai,...,  An  is  hypocrisy  for  the  formula  Ai  0  . . .  0  An  ; 

p  A  discharged  formula  [A]  is  hypocrisy  for  ?A  ; 

►  If  A;£  ,  An  are  hypocrisy  for  formulas  Ai,...,  An,  then  the  sequent  i-  A^ ; , . , ;  An 

is  hypocrisy  for  the  formula  Ai  An. 


i-r;A  \-T-,A[Bla] 

-  (for  all :  a  is  not  -  (there  is) 

i-r;VaA  free  in  V) 

2.4  The  expressive  power  of  LLL 

Our  goal  here  is  to  prove  that  poly  time  functions  can  be  represented  in  LLL. 
This  can  be  established  by  various,  means.  We  adopt  the  simplest  (maybe  not 
the  most  elegant)  solution,  namely  to  encode  poly  time  Turing  machines  in  an 
intuitionistic  version  of  LLL,  ILLL.  There  will  be  a  forgetful  function  of  ILLL 
into  system  !F  (with  conjunction),  hence  the  ultimate  interpretation  will  be  in 
system  T ^  in  which  the  representation  of  data,  algorithms,  is  quite  familiar. . . 

2.4.1  The  system  ILL 

The  language  of  ILL  is  based  on  the  connectives  0,  &,  -o, !,  §,  and  second-order 
quantification.  The  sequents  of  ILL  are  of  the  form  . . . ;  An  i-  5,  where 
Ai,..  .  ;An  are  blocks,  and  B  is  a  formula.  The  formulas  I*'  =!*"!  are  allowed 
in  the  blocks,  although  they  are  not  part  of  the  language  of  ILLL  The  rules 
of  ILLL  are  those  that  remain  correct  when  we  translate  A^ ; . . . ;  An  »-  5  as 

I-  A^"^; . . . ;  An"^;  5- 

The  forgetful  functor  (erasizre)  from  ILLL  into  T  is  defined  eus  follows  : 

►  To  a  formula  A  of  ILLL  we  associate  A~ ,  a  type  of  as  follows  :  !  and  §  are 
erased,  (g)  and  h  are  replaced  with  A,  — o  is  replaced  with  =>,  variables  and 
quantifiers  are  unchanged  ; 

►  To  a  sequent  S  of  ILLL  we  associate  a  sequent  S~  of  second  order  proposi¬ 
tional  logic  : 

8.  This  is  due  to  the  techniced  restrictions  on  the  !-rule  ;  should  ILLL  be  developed  for  its 
own  salce,  these  special  formulas  should  be  replaced  with  markers. . . 
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-  In  5  remove  all  discharges  [•]  ; 

-  In  5  replace  all  semicolons  by  commeis  ; 

-  In  5  replace  all  formulas  of  LLL  by  their  erasure  ; 

-  In  5  remove  all  formulas  1^. 


Typically  the  erasure  of  A,  S,  !!1;  [1];  [C];  D,  £*  i-  F  will  be  interpreted  as 
A~ ,B~ ,C~,D~ ,E~  h- F~ . 


►  To  a  proof  11  of  5  we  associate  a  proof  11"  of  5" ,  or  equivalently  a  term  of 
system  F.  Typically  a  proof  of  A,  B,  !!1;  [1];  [C];  D,Ei-  F  will  induce  a  term 
depending  on  variables  v^w,x,y,z  : 

V  :  A~ ,w  :  B~  yX  :C~,y  :D~^z  :E~t-t  :  F~ 

The  interpretation  is  straightforward  ;  observe  that  the  erasure  of  is  un¬ 
problematic,  since  indeed  deals  with  weakening,  i.e.  dummy  variables. 

The  basic  idea  behind  the  erasure  is  that  (intuitionistic)  linear  logic  (light  or 
not)  can  be  viewed  as  a  more  refined  way  to  speak  of  implication,  conjunction, 
erasing  reuse.  These  refinements  are  not  taken  into  account  in  intuitionistic 
logic,  and  the  forgetful  functor  collapses  the  two  conjunctions,  ignores  exponen¬ 
tials  (and  therefore  destroys  1*,  a  very  subtle  handling  of  weakening).  This  is 
reflected  in  the  translation  of  the  formulas  and  also  of  the  sequent s,  where  the 
additive,  multiplicative  and  exponential  layers  (represented  by  •]  are 

collapsed  into  a  comma.  When  we  shall  represent  data  and  algorithms  in  ILLL, 
we  shall  implicitly  refer  to  their  forgetful  image  in  F.  It  goes  without  saying  that 
the  notion  of  reduction  to  be  defined  in  LLL  is  compatible  with  the  notion  of 
reduction  of  F,  so  that  the  only  consideration  of  the  forgetful  images  matters.  In 
what  follows  the  most  important  functions  are  represented  in  details  ;  we  assume 
that  the  reader  is  most  familiar  with  system  F^  the  Curry- Howard  isomorphism 
which  identifies  natural  deduction  with  typed  A- terms,  and  therefore  that  he 
has  no  problem  to  synthesize  the  A-term  associated  with  a  proof  in  second-order 
intuitionistic  sequent  calculus. 


2.4.2  Representation  of  tally  integers 
Integers  We  define  the  type  int  of  tally  integers  by 


int  =  Va.  !(a  — o  a)  — o  §(«  -o  or) 


The  tally  integer  n  is  obtained  as  follows  : 

Of  I-  a  a  I-  a 


a— o  or;  0:1-0;  ai-o; 


0;  -0  0;; 

a  —0  a;  0;  1-  0; 

o;  -0  a; . . 

. ;  0  -0  a;  0;  1-  0; 

a  -0  a; . . . 

;  a  — 0  a  1-  0;  —0  0: 

[a-oa];...;| 

[a  —0  0;]  1-  §(a  —0  a) 

[a  -0  a]  1-  §(q;  -0  0;) 

!(o;  —0  a)  1-  §(a  -0  a) 

i-!(o  —0  0;)  —0  §(0;  -0  0:) 

1-  Va.  !(o;  — o  o:)  — o  §(a  -o  o;) 


It  is  immediate  that  int  =  Vo;,  (o;  o;)  (a  o;),  and  that 

(72)“  =  ,x{x  . .  .(ic(t/)) . . .)  . 


Addition 

Addition  is  proof  +  of  int,  int  1-  int  obtained  as  follows  : 

o;  I-  o;  ot\-  a 
o;— oa;ai-a  ai-o; 

o;— oo;;q:— oa;o;i“a 

a  — o  o:;  o;  — o  a  I- o;  — o  a  a  — o  o:  1- o;  — o  a 

§(a  — o  o;);  §(a -o  o)  I- §(a -o  o;)  [a  ~o  o;]  i-!(o; -o  o;)  o; -o  a  1- o; -o  o; 

inter;  §(q;  ck);  [a  -o  a] »-  §(a  — o  a)  [o;  -o  a]  i-!(q;  -o  a) 

inta;intcr;[Q;-oo;];[o;-oa]  1-  §(o;-oa) 
inter;  inter;  [a  -o  a]  1-  §(a  -o  a) 
inter ;  inter  ■“  l^ta 
int;  inter  *- 
int;  int  1-  inter 
int;  int  1-  int 


with  inter  =!(a  -o  a)  -o  §(a  -o  a).  It  is  immediate  that  the  erasure  of  +  is  the 
usual  representation  of  addition  in  !F.  . 
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Multiplication 

Multiplication  is  a  proof  x  of  !int;  int  i-  §int  : 

:o  :  + 

int  I-  int  i-  int  int;  int  i-  int 
int  -o  int  i-  int  int  i-  int  — o  int 
§(int  -o  int)  i-  §int  [int]  i-!(int  -o  int) 

[int];  i-  §int 

[int];  int  I-  §int 
!int;  int  i-  §int 

It  is  immediate  that  the  erasure  of  x  is  the  usual  representation  of  multiplication 
in  T . 

Iteration 

The  principle  of  Heraiion  is  derivable  :  if  F  is  a  block  and  not  discharged, 
then  from  a  proof  of  F  i-  A  -o  A  and  a  proof  of  A  i-  ^A,  one  can  derive 
[F];  A;int  I- §A  : 


F  I—  — o  a  §j4  I—  A  I—  §>1 

[F]i-!(A^A)  ^§A-o§Ai-§A 
[F];  A;intA  §A 
[F];  A; inti- 

It  is  immediate  that  the  erasure  of  iteration  is  the  usual  representation  of  itera¬ 
tion  in  T  ;  however,  very  few  actual  iterations  of  T  can  be  obtained  this  way. 
The  types  list^,  to  be  defined  below,  have  similar  primitives,  including  a  notion 
of  iteration. 

Coercions 

Observe  that  any  sequent  ; . . . ;  1*”;  F  i-  A  can  be  replaced  with  1*;  F  i-  A, 
provided  k  ^  We  shall  content  ourselves  with  a  weaker  typing  of 

integers,  namely  F  i-  ^‘^int  (in  general  p  =  but  we  do  not  mind  about  the 
actual  value  of  p.  A  n-ary  function  from  integers  to  integers  will  be  given  a  type 
F;int; . .  .;int  1-  §^int. 

The  successor  function  is  naturally  typed  1-  int  1-  int,  which  can  be  replaced 
with  !*int  i-!^int,  and  therefore  by  1  i-!^int  -o  l^int.  The  integer  0  can  be 
given  the  type  1 1-  int,  hence  the  type  1*  i-!*int,  and  also  the  type 
l*+i  i_  §!*int.  We  are  in  position  to  apply  iteration  and  we  get  a  function  which 
is  typed  [1];  l*+^;int  1-  §!*int,  which  can  be  replaced  with  l*'‘'^;int  1-  §!^int. 
This  function  is  essentially  the  identity  on  integers,  but  it  changes  the  type,  and 
we  call  it  a  coercion. 
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In  a  similar  way,  we  can  define  coercions  of  type  int  i-  §^!^int,  when  p  ^  0. 
An  immediate  consequence  is  that  the  multiplication  can  be  given  a  more  even 
type  :  replace  !mt;int  i-  §int  with  §!int;§int  i-  §^int,  and  compose  with  the 
coercions  l^;inti-  §!int  and  !l;inti-  §int,  in  order  to  get 
P;  !1;  int;  int  i-  §^int,  which  can  be  simplified  into  1^;  int;  int  i-  §^int. 

It  is  then  easy  to  see  that,  if  /(xi, . . . , Xn)  and  p(y, yi, . . . , 2/m)  have  been  at¬ 
tributed  types  F;  int;...;  int  i-  §^int  and  F;  int; . . .  ;int  i-  §^int,  then  the 
function  g(f{xi , . . . ,  a^n),  yi , . .  • ,  Pm)  can  be  given  a  similar  type,  namely 
F+^ ;  int; . . . ;  int  i-  §^'^^int.  In  other  terms,  all  polynomials  in  which  each  vari¬ 
able  occurs  exactly  once  can  be  typed. 


Weakening  and  contraction 

In  order  to  get  all  polynomials,  we  must  be  able  to  represent  dummy  dependen¬ 
cies,  and  repetition  of  variables,  i.e.  weakening  and  contraction  for  int. 
Weakening  can  be  defined  as  a  sum  with  a  function  which  is  identically  0  : 

a  — o  a  I- a —o  a  at-a 

§(a  — o  a)  I- §(o -o  or)  i- a  — o  a 

§(a -o  a);  !(a  “O  a)  I- §(a -o  a)  [1]  i-!(q!  ^  a) 
[l];inta;!(Q^-oa)i-  §(Q;--oQf) 

[l];inta  I-  int  a 


etc.  . 

Contraction  is  obtained  by  composition  with  a  diagonal  map,  i.e.  the  function 
diag{n)  =<  n,  n  >.  For  this  observe  that  the  successor  induces  a  map  of  type 
int  (g)  int  i-  int  (g  int  (corresponding  to  the  function 

/(<  n,  n  >)  =<  n  H-  1,  n  4-  1  >,  which  can  be  retyped  1;  int  (g  int  i-  int  (g  int, 
and  0  induces  an  object  of  type  int  0  int.  By  iteration,  we  get  a  function  of 
type  [l];int  i-  §  int  0  int,  corresponding  to  the  map  f{n)  =<  n,n  >.  Now,  if 
compose  with  the  diagonal  map,  it  is  clear  that  we  can  identify  variables  (in 
general  the  integer  k  will  increase). 

So  all  polynomials  can  be  given  a  type  F;  int; . . . ;  int  i-  §^int,  and  we  can  even 
fix  the  value  of  k  when  the  degree  is  known. 

Similar  weakening  and  contraction  maps  are  available  for  the  types  list^  to  be 
defined  below,  in  particular  for  bint. 


The  predecessor 

Last,  but  not  least,  we  must  type  the  predecessor,  i.e.  the  function  pred  such 
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that  prec?(0)  —  0,p(n  +  1)  =  n.  The  predecessor  gets  the  type  !l;int  i-  int  : 


l;a  t-  a 

at- a  at- a  l,a—oa\a}  t-a 


at-  a  at-  a 


a  —o  a\at-  a 


l,a  -o  a;a^  t- 


a^  — o  a  I-  a 

a^  —o  a^  t-  a  —o  a 


1,  a  “O  a;  I- 
1,0;— oai-  0^—00 


a  -oa't-a—oa  [1];  [a  — o  a]  i“!(a^  — o  o^) 

§(0^  — o  a^)  I-  §(0;  — o  a)  !1;  !(q;  — o  o)  i-!(q;^  -o  a^) 

!1; intent;  !(qj  -o  o;)  i-  §(o  -o  a) 

!l;int^t  •-  int^ 

etc.,  with  a^  =  a  ka.  .  The  basic  idea  is  to  iterate,  instead  of  /  of  type  a—oa, 
the  function  f  in  a^  t-  a^  such  that  f'{x)  -<  x,  f(x)  >  Eventually  the  first 
projection  of  the  result  is  kept.  Similar  functions  for  ib-lists  can  be  defined. 


2.4.3  Some  data  types 

Familiar  data  types  as  well  as  the  basic  operations  on  them  can  be  represented 
in  LLL.  We  shall  only  need  a  type  with  n  elements  bool^  and  the  type  of  lists 
of  tokens  taken  among  m  tokens,  list”^. 

Booleans 

The  type  bool^  is  defined  as  Vo.  §(o  .  .0  — o  o)  ;  there  are  k  occurrences 
of  o  to  the  left,  and  we  agree  on  some  (irrelevant)  bracketing  convention.  In 
particular,  bool^  (written  more  simply  bool)  is  Vo.  §(o  o  -o  o).  Its  erasure 
Vo.  o  A  o  =>  o  is  one  of  the  standard  representations  of  booleans  in  !F.  We  can 
define  proofs  bi,. .  .,bfc  of  bool,  by  starting  with  one  of  the  k  canonical  proofs 
of  o  &  . .  .0  I-  o,  and  ending  with  -o,  §  and  V-rules.  Typically  the  boolean 
«  false  »,  b2  is  : 

01-0 

- &2 

akat-a 

t-  a  k  a  -o  a 

t-  §(o  &:  o  -o  q;) 


whose  erasure  is  the  standard  term  Ao  Xx°‘^°‘  7r2(x),  which  represents  «  false  » 
in  T, 


9.  /'is  not  quite  linear  in  /,  which  is  reflected  by  the  [l]  in  the  sequent  [1];  a  -o  a  i-  at  _o 
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If  then  else 

We  can  give  a  type  ;  booP;  ;  A  i-  A  to  the  fc-ary  version  of  «  if. . .  then 
. . .  else  ...  »  ,  when  A  is  a  data  type  (it  works  when  A  is  a  boolean  type  or 
a  type  of  lists).  We  give  an  example  when  A  is  int  and  k  =  2,  i.e.  we  try  to 
”type”  the  function  f{true,  n,  m)  =  n,  f  {false,  n,  m)  =  m. 

ai-a  och-a  oct-a  ai-a 

q:;q!— oai-a  a]  a -oat- a 

a;  a  *“0  a;  1 1- a  or 

at-a  a\\,a-o  a\\,a-o  at- a  ha 
a  &  a;  1,  a -o  a;  1,  a -o  a;  a  Of 
a  &  a;  1,  a  “O  a;  1,  a  — o  a  a  — o  O'  a-oat-a-^-oa 

[l];boolor; §(a  — o Qr);§(a  — o  O')  I- §(a -o  a)  [a -o  a]  »~!(a -*o a)  a-oai-a-oa 
[1];  boolcrl  intai  §(a  -o  a);  [a  -o  a]  i-  §(a  -o  a)  [a-o  or]  i-!(a  -o  a) 

[1];  boola;  intal  inta;  [a  -o  a]  •-  §(a  -o  a) 

[1];  boolaj  inta;  inta;  K®  -o  «) »-  §(a  -o  a) 

[1];  boola;  inta;  inta  t-  inta 


etc.,  with  boola  =  §(a  &  a  -o  a).  Weakening  and  contraction  on  bool^  can  be 
defined  in  terms  of  generalized  «  if. . .  then - else  ...  ». 


Lists 

We  define  list^  to  be  Va.  (!(q;  — o  a)  -o  (. . .  — o!(a  -o  a) . . .))  -o  §(a  — o  a),  with  k 
occurrences  of  !(a  -oa)  to  the  left.  So  list^  is  just  int,  and  list^  is  abbreviated 
into  bint  (binary  integers). 

We  discuss  the  type  bint,  but  our  discussion  applies  to  any  type  list^.  First 
we  observe  that  the  empty  list  emptylist  and  more  generally  any  finite  list  of 
digits  0  and  1  can  be  encoded  by  a  proof  of  bint.  This  is  more  or  less  obvious, 
since  bint"  is  the  usual  .7^- translation  of  binary  lists.  Concatenation  of  lists, 
can  be  represented  by  a  proof  of  bint;  bint  i-  bint,  which  is  basically  a 
binary  version  of  +,  and  that  we  therefore  skip.  In  particular  the  two  successor 
functions  •  ^  0  and  •  1  can  both  be  given  the  type  bint  i-  bint. 

An  important  function  is  the  «  kind  of  list  »,  of  type  !1;  bint  i-  bool^.  On  the 
empty  list  it  yields  the  value  bi,  on  a  list  ending  with  0,  it  yields  the  value  b2, 
and  on  a  list  ending  with  1,  it  yields  the  value  ba  :  let  a*  =  (a  &  a)  a,  and 
introduce  three  proofs  11;,  11^  and  ILh  of  a*  t-  a*  respectively  corresponding  to 
the  functions  f{x)  —«  7ri(7ri(a;)),  7ri(7ri(a?))  >,  7ri(7ri(a:))  >, 
g{x)  =«  7r2(7ri(ar)),7r2(7ri(a;))  >,  7r2(7ri(a;))  >, 
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h{x)  =«  Tr2(x),Tr2(x)  >,7r2{x)  >, 

:ny 

or  I-  a  a*  I-  a*.  ; 

a*  -o  Of  I-  Of*  —o  a  a*  I-  or*. 

§(a* -o  a*)  I- §(«* -o  Of)  [1]  i-!(a* -o  a*) 

-o  a*)  -o  ^  a*)  §(a*  ^  g)  [1]  i-!(a*  ^  g*) 

[l];binta*  i-  §(a*  -og) 

!l;  bint  i-  §(q!*  — o  a) 

!1;  bint  i-  bool 

with  binta  =!(»  —o  a)  —o  (!(q;  — o  a)  — o  §(a  — o  a)). 

Among  the  functions  connected  with  list^  are  all  the  functions  list^,  of  type 
list^  I-  list^',  induced  by  a  map  /  from  {1, . . . ,  to  k*}.  They  are 

easily  defined,  mainly  by  structural  manipulations.  Three  important  examples  : 

►  The  (unique)  function  list^  from  bint  to  int  identifies  the  two  digits,  and 
produces  a  tally  integer  :  it  will  be  used  for  the  length  of  the  input  of  a 
Turing  machine  ; 

►  When  ^  ^  2,  the  function  from  bint  to  list^  that  identifies  a  binary  integer 
with  a  Ar-list  :  it  will  be  used  for  the  input  tape  of  a  Turing  machine  ; 

►  When  Ar  ^  2,  the  function  from  list^  to  bint  that  replaces  any  digit  distinct 
from  0, 1  with  0  :  it  will  be  used  for  the  output  of  a  Turing  machine. 

2.4.4  Polytime  functions 
Turing  machines 

Consider  a  (deterministic)  Turing  machine  using  p  symbols  and  with  q  states. 
The  current  configuration  can  be  represented  by  three  data  : 

►  a  list  dealing  with  the  leftmost  part  of  the  tape  (up  to  the  position  of  the 
head) 

►  a  list  dealing  with  the  right  part  of  the  tape,  in  reverse  order 

►  the  current  state 

The  type  TurP'^  =  listP  0  listP  0  bool^  can  therefore  be  used  to  represent 
any  configuration  of  the  machine.  The  instructions  of  the  machine  depend  on 
reading  the  last  symbol  of  one  list,  (including  testing  whether  or  not  a  list 
is  empty),  and  also  depend  on  the  current  state.  From  what  precedes,  it  is 
possible  (by  eventually  adding  new  instructions  so  that  the  machine  can  never 
stop),  to  represent  a  Turing  machine  by  a  proof  of  !1;  TurP’^  i-  TurP’^  :  just 
use  successors,  predecessors,  «  kind  of  list  »,  and  generalized  «  if  . . .  then  . . . 
else. . .  ». 


a*  I-  a*. 
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Inputs  and  outputs 

We  assume  that  our  inputs  are  binary  integers,  i.e.  that  the  digits  0  and  1 
belong  to  the  p  legal  symbols  of  the  tape.  The  input  (initial  configuration)  can 
therefore  be  expressed  by  means  of  a  map  of  type  bint  i-  TurP'^  which  maps 
a  binary  list  s  into  the  3-tuple  <  s,emptylist,  a  >,  where  cr  is  the  initial  state. 
When  the  expected  runtime  of  the  machine  is  over,  we  may  also  decide  to  read 
off  the  output,  i.e.  we  need  to  represent  the  map  /(<  s,f,r  >)  =  s',  where  s'  is 
obtained  from  s  by  replacing  any  symbol  distinct  from  0, 1  by  0.  Such  a  function 
is  easily  obtained  by  means  of  a  function  list^  and  of  the  weakening  facilities  on 
our  data  types. 


Run  of  a  Turing  machine 

Assume  that  we  are  given  a  time  0  (represented  by  a  tally  integer  of  type  int), 
an  initial  input  s  of  type  bint,  and  a  Turing  machine  of  type 
!l;TurP'^  i-  TurP'^.  Then  running  the  machine  for  6  steps  from  the  initial 
input,  can  be  represented  by  means  of  an  iteration.  As  a  function  of  0,  s  it  may 
receive  the  type  l^;int;  §bint  i-  §TurP>^  and  therefore  (using  the  coercion  map 
11; bint  i-  §bint)  also  the  type  1^;  int; bint  i-  §TurP>^).  The  result  at  time  6 
(if  we  stop  the  machine  after  6  steps)  can  be  written  as  a  function  of  0,  s  of  type 
1^ ;  int;  bint  i-  §bint. 


Poly  time  machines 

A  polytime  machine  is  machine  with  a  polynomial  clock,  which  stops  after 
P(|j(s))  steps,  where  tt(s)  is  the  size  of  the  input,  and  P  is  a  given  polyno¬ 
mial  ;  when  P{i{s))  steps  have  been  executed,  then  we  print  out  the  result. 
Now  observe  that  P  can  be  given  a  type  l*;int  i-  §*int,  and  using  the  (unique) 
map  list^  from  bint  into  int,  the  function  P'(s)  =  P(tl(s))  can  be  given  the 
type  l*;bint  i-  §*int.  By  composition  with  the  runtime  function,  we  get  the 
type  l^'*'^;bint;  §*^bint  i-  §*''*'^bint  to  represent  the  function  <p{s,s')  which  is 
the  result  of  the  computation  after  P'(tt(s))  steps  with  the  input  s'.  Using  the 
contraction  facility  on  bint)  we  can  make  s  =  s'  and  replace  this  type  with 
1^+3;  bint  h~  §*^‘^^bint.  If  we  insist  on  having  the  same  integer  on  both  sides, 
we  can,  using  the  coercion  of  type  §*'^^bint  §*^**‘^bint,  replace  this  type 

with  1*+^;  bint  I-  §*^‘‘'^bint. 

2,4.5  The  representation  theorem 
Theorem  1 

Any  polytime  function  from  binary  lists  to  binary  lists  can  be  represented 
in  LLL  as  a  proof  of  a  formula  1*^;  bint  §*^bint 

Proof.  —  This  is  obvious  from  what  precedes.  The  algorithm  can  be  executed 
in  P,  but  also  as  a  proof-net,  in  which  case  the  output  is  a  proof-net  with  con¬ 
clusions  §^bint;  and  the  which  eventually  comes  from  0-ary  ±-links,  can 
be  ignored.  ^ 
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2.4.6  The  user’s  viewpoint 

Let  us  admit  that  this  works,  without  being  especially  friendly.  Since  this  paper 
is  concerned  with  showing  that  LLL  is  intrinsically  polytime,  this  subsection 
was  concerned  with  a  rather  marginal  question  :  to  show  that  it  was  strictly 
polytime,  i.e.  that  any  polytime  function  could  be  typed  inside  the  system. 
So  we  didn’t  care  much  about  the  potential  users  of  such  a  system.  Surely 
this  practical  aspect  should  be  developed,  under  the  form  of  a  typed  A-calculus, 
analogous  to  system  T.  The  best  would  be  system  of  pure  A-calculus  with  typing 
declarations  in  ILLL. 

But  this  is  not  the  only  possibility  :  a  less  conservative  option  would  be  to  exploit 
the  classical  symmetries  of  LLL,  which  have  a  lot  of  interesting  consequences 
(for  instance,  using  the  fact  that  a  — o  o:  is  isomorphic  to  or  -r-o  a,  which  induces 
an  isomorphism  between  bint^x  and  binta,  we  get  the  proof  : 

bint^x  I-  binta 
bint  I-  binta 
bint  I-  bint 

whose  action  is  to  reverse  a  list). 


3  PROOF-NETS  FOR  LLL 

Cut-elimination  in  sequent  calculus  is  unmanageable  -especially  in  presence  of 
additive  features-  :  too  many  permutations  of  rules  occur,  and  the  counting  of 
these  permutations  blurs  the  actual  complexity  of  the  process.  This  is  why  we 
choose  to  use  proof-nets  to  prove  the  main  theorem  of  this  paper.  Our  basic 
reference  will  be  [2]  where  the  proof-net  technology  is  expounded.  We  shall 
therefore  content  ourselves  with  modifying  the  definitions  of  [2]  so  as  to  take 
care  of  the  specificities  of  LLL.  We  adopt  the  definitions  and  conventions  of  this 
paper,  in  particular  we  shall  very  often  speak  of  formulas  to  mean  ’’occurrences 
of  formulas” .  We  shall  ignore  the  additive  constants  T  and  0  on  the  double 
grounds  that  they  play,  little  role  and  that  they  can  be  handled  anyway  by 
means  of  second-order  definitions  in  case  we  badly  insist  to  keep  them.  This  will 
save  a  lot  of  inessential  details. 


3.1  Proof-nets  with  multiplicative/additive  conclusions 

We  first  liberalize  the  condition  about  the  weights  of  conclusions  in  definition 
3  of  [2].  Let  r  =  [A];  A An  be  a  sequent.  Then  a  proof-structure  will 
be  declared  to  have  the  conclusion  F  when  its  conclusions  are  the  formulas 
(discharged  or  not)  listed  in  F  and  furthermore,  for  each  Aj  the  sum  of  the 
weights  of  the  formulas  of  Aj  is  equal  to  1.  This  is  equivalent  to  saying  that, 
after  applying  ad  hoc  0-links  to  the  formulas  of  Aj,  then  we  obtain  a  proof- 
structure  in  the  sense  of  section  3  of  [2]. 

We  consider  the  following  exponentials  links  : 
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►  The  ?-link,  with  n  unordered  premises,  which  are  all  occurrences  of  the  same 
discharged  formula  [i4],  and  with  conclusion  ?A  ; 

p-  The  !-box,  which  is  a  generalized  axiom  whose  (unordered)  conclusions  are 
[Ai], . . . ;  [An]]  \B.  This  link  is  called  a  box  because  in  order  to  use  it,  one  has 
to  give  a  proof-net  0  whose  conclusions  are  , . . . ,  An ;  5  ;  our  conventions 
about  proof-structures  imply  that  n  is  nonzero.  A  pictural  representation  of  a 
box  is  precisely  a. . .  box  whose  contents  is  0  and  below  which  the  conclusions 
of  the  link  are  written  ; 

p  The  §-box,  which  is  a  generalized  axiom  whose  (unordered)  conclusions  are 
[Ai]; . . . ;  [An];  §An+i; . . . ;  §An+m.  This  link  is  called  a  box  because  in  order 
to  use  it,  one  has  to  give  a  proof-net  0  whose  conclusion  is  a  sequent  F;  A 
without  discharged  formulas,  and  such  that  the  formulas  occurring  in  F  are 
exactly  Ai, . . .,  An,  and  A  is  §An+i; . .  .;§An+m.  A  typical  example  is  that 
of  a  proof-net  with  conclusions  A,  B]  C]  D,  E,  F]  G]  H,  which  can  be  used  to 
form  a  §-box  with  conclusions  [A];  [B]]  [C]]  [D];  [E]][F]]^G]^H ,  but  also  a 
§-box  with  conclusions  [A] ;  [B] ;  §C ;  [73] ;  [E] ;  [F] ;  §G ;  §77 . 

Weights  are  subject  to  the  usual  conditions  ;  moreover 

p  A  discharged  formula  is  the  conclusion  of  exactly  one  link,  i.e.  one  box  ; 

►  If  1/  is  a  ?-link  with  premises  [Ai], . . . ,  [An]  (occurrences  of  the  same  dis¬ 
charged  formula),  then  w{L)  ^  u;([Ai])  for  i  =  1, . .  .,n  ;  remember  that  a 
default  jump,  i.e.  a  formula  B  such  that  w{B)  ^  n;(L)  must  be  provided  with 
the  link. 

The  condition  for  being  a  proof-net  is  defined  in  the  obvious  way  :  once  a  valu¬ 
ation  xp  has  been  selected,  one  builds  a  graph  whose  vertices  are  those  formulas 
A  such  that  'tp(w{A))  =  1.  The  edges  are  selected  as  in  [2]  ;  moreover 

p  For  any  ?-link,  one  draws  an  edge  between  the  conclusion  of  the  link  and  any 
premise  of  the  link  which  a  vertex  of  the  graph,  or  with  the  default  jump  B 
(this  is  crucial  in  case  no  premise  of  the  link  is  a  vertex  of  the  graph)  ; 

p  For  any  box  with  conclusions  Ai,...,An,  one  draws  an  edge  between  Ai 
and  A2,  A2  and  A3,. . .  An-i  and  An.  The  choice  of  edges  depends  on  an 
ordering  of  the  conclusions  of  the  box,  but  any  other  ordering  would  produce 
an  equivalent  graph. 

Observe  that  since  boxes  are  built  from  proof-nets,  our  condition  indeed  means 
that  a  proof-structure  is  a  proof-net  iff  it  is  a  proof-net  when  we  consider  its 
boxes  as  proper  axioms,  and  if  the  contents  of  its  boxes  are  in  turn  proof-nets, 
etc. 

3.2  Sequent ializat ion  for  LLL 

We  must  first  define  what  it  means  for  a  proof  in  sequent  calculus  to  be  a 
sequeniializaiion  of  a  proof-net.  This  is  done  without  problem,  following  the 
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lines  of  [2],  We  only  need  to  be  careful  about  the  structural  maintenance  : 
typically  certain  formulas  of  i-  T  are  not  present  in  the  proof-net,  because  they 
would  receive  the  weight  0.  This  is  the  case  inside  blocks,  and  for  discharged 
formulas.  We  can  state  the  : 

Theorem  2 

Proof-nets  are  sequentializable,  i.e.  every  proof-net  is  the  sequentialization 
of  at  least  one  sequent  calculus  proof. 

Proof.  —  By  induction  on  the  depth,  i.e.  the  maximum  nesting  of  boxes.  If 
we  assume  that  the  inside  of  all  boxes  is  sequentializable,  since  the  rules  for  the 
formation  of  boxes  are  the  same  as  the  rules  for  !  and  §,  then  we  are  left  with 
the  problem  of  sequentializing  a  usual  proof-net  with  boxes,  a  question  solved 
in  [2],  section  3.  □ 


3.3  Cut-elimination  for  LLL 

Since  this  proof  is  rather  delicate,  we  suggest  to  first  understand  it  in  the  case 
without  additives.  Hence  there  is  notion  of  weight,  the  !-boxes  have  exactly  two 
conclusions,  all  cuts  are  ready,  and  all  exponential  cuts  are  special.  Moreover 
the  notion  of  proof- net  in  this  case  is  akin  to  the  more  familiar  multiplicative 
case. 


3.3.1  The  size  and  depth  of  a  proof-net 
Definition  4 

The  size  jt(L)  of  a  link  L  is  dehned  by  : 

►  if  L  is  an  identity  link,  jt(L)  =  2  ; 

►  if  L  is  a  cut-link,  i(L)  =  0  ; 

►  if  L  is  an  exponential  box  constructed  from  a  proof-net  with  conclusion 
r,  then  jt(jL)  =  1  -f-  s,  where  s  is  the  number  of  semi-columns  in  T  ; 

►  otherwise  tt(Zr)  =  1. 

The  size  }(0)  of  a  proof-net  0  is  the  sum  of  the  sizes  of  the  links  occurring 
in  it,  including  what  (hereditarily)  occurs  inside  the  boxes. 

Definition  5 

The  depth  ^(0)  of  a  proof-net  0  is  the  maximum  nesting  number  of  boxes 
in  0.  The  depth  of  a  formula  A  (denoted  dA  or  dA/B)  is  the  number  of 
boxes  containing  it  :  typically,  if  0  consists  of  a  sole  box  B  made  from  a 
proof-net  0',  then  the  conclusions  of  B  have  depth  0,  whereas  the  depth  of 
a  formula  A  of  0'  is  given  by  dA/Q  =  dA/B'  + 1 .  One  similarly  defines  the 
notion  of  depth  of  a  link  :  typically  in  the  case  just  considered,  the  box  gets 
the  depth  0,  whereas  dL/B  =  dL/B’  A  1  for  all  links  L  occurring  inside  B. 
Finally  we  define  the  partial  size,  also  called  d-size,  #^(0)  to  be  the  sum 
of  the  sizes  of  links  of  depth  d  in  0,  so  that  #(0)  =  jti(0)  -f  . . .  -f  jtn(0), 
where  n  is  the  depth  of  0. 
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These  definitions  have  been  chosen  because  of  their  relevance  to  cut-elimination. 
But  what  about  the  relevance  of  our  size  w.r.t.  the  actual  size  of  a  proof-net  ? 

►  The  size  of  a  link  is  almost  the  number  of  its  conclusions.  In  0,  define 
a  function  /  as  follows  :  if  A  is  not  discharged,  \etf{A)  be  any  link  with 
conclusion  if  [^]  is  discharged,  then  it  is  the  conclusion  of  a  box,  and  A 
occurs  (undischarged)  inside  the  box,  and  we  set  f{[A])  =  f{A).  It  is  easy 
to  see  that  L  occurs  in  the  range  of  /  at  most  twice  the  size  of  T,  hence  the 
number  of  formulas  in  ©  is  bounded  by  2tt(L). 

►  Cut-links  do  not  contribute  to  the  size  ;  however  if  A  is  the  premise  of  such 
a  link,  then  A  is  the  conclusion  of  another  link,  and  it  is  easy  to  see  that  the 
number  of  cuts  cannot  exceed  the  size. 

►  The  actual  size  of  a  net  as  a  graph  is  therefore  linear  in  the  official  size. 
Good  News  !  .  However  we  are  not  done  since  the  net  also  involves  the  boolean 
weights.  But,  as  observed  in  [2],  these  weight  can  be  replaced  with  a  structure 
of  coherent  space  between  the  links  and  therefore  the  size  of  the  missing 
structure  is  quadratic  in  the  ofl&cial  size  of  the  net. 

►  Finally  the  size  does  not  take  into  account  the  actual  sizes  of  formulas.  Here 
vary  little  can  be  done,  especially  in  presence  of  quantifiers.  The  most  natural 
viewpoint  is  to  see  the  formulas  as  comments,  which  are  erased  at  runtime,  in 
the  same  way  that  the  actual  execution  of  a  typed  A-term  is  the  execution  of 
its  erasure,  i.e.  of  the  underlying  pure  A-term.  In  other  terms,  we  work  with 
a  kind  of  interaction  net  a  la  Lafont,  see  [6]. 

This  should  be  enough  to  convince  one  that  the  polynomial  bounds  obtained 
below  actually  induce  a  polytime  algorithm.  Concretely,  as  explained  in  [2],  the 
substitutions  occurring  during  the  additive  steps  are  delayed  and  those  occurring 
during  the  quantifier  steps  are  not  performed  (they  can  be  stored  in  some  auxil¬ 
iary  memory).  If  the  final  result  should  be  without  additives,  then  the  additive 
substitutions  can  be  done  at  the  end,  producing  a  cleansing  of  the  graph  (all 
weights  become  0  or  1).  If  the  final  result  is  also  free  from  any  kind  of  existential 
quantifiers,  then  the  formulas  can  be  synthesized  in  an  obvious  way,  and  we  have 
no  use  for  our  stack  of  substitutions. 

3.3.2  Cut-elimination  :  the  general  pattern 

We  shall  define  a  lazy  cut-elimination  which  terminates  in  polytime.  The  result 
of  the  procedure  (which  is  Church-Rosser)  is  cut-free  only  in  certain  cases,  but 
this  is' enough  for  us. 

Let  us  call  a  cut  exponential  when  the  cut-formulas  begin  with  exponentials  and 
both  premises  are  conclusions  of  exponential  links.  For  non-exponential  ready 
links,  the  paper  [2]  defines  a  linear  time  cut-elimination  procedure  :  each  step  of 
this  basic  procedure  strictly  shrinks  the  size  of  the  proof-net  (and  this  remains 
true  with  our  specific  measurement  of  size)  .The  pattern  is  as  follows  : 
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►  In  a  preliminary  round  we  apply  the  basic  procedure  at  depth  0,  which  induces 
a  shrinking  of  the  proof-net  at  depth  0,  the  other  sizes  staying  the  same  ;  then 
the  real  things  begin 

►  In  a  first  round  we  work  at  depths  0  and  1  ;  at  depth  1  only  the  basic 
procedure  is  allowed,  whereas  only  certain  exponential  cuts  are  removed  at 
depth  0.  If  the  original  partial  sizes  were  sq,  . .  .,Sd,  then  the  new  sizes  after 
completing  the  procedure,  will  not  exceed  5o,  so^i, .  • . ,  (and  the  depth 
does  not  increase). 

►  In  a  second  round  we  apply  a  similar  procedure  at  depths  1  and  2  ;  this 
procedure  fires  no  new  reduction  at  depth  0,  so  that  after  completing  this 
second  round,  our  partial  sizes  will  not  exceed  5o,  sqSi,  So^i52,  • . . , 

and  the  depth  still  not  increases. 

►  The  round  occurs  at  depths  c?  -  1  and  d.  When  it  is  completed,  noth¬ 

ing  more  can  be  done  (in  the  lazy  case,  we  shall  be  cut-free).  The  depth 
of  the  proof-net  is  still  at  most  d  (it  can  diminish  in  the  very  unlikely  sit¬ 
uation  of  erasure  of  a  deeply  nested  box),  and  the  sizes  are  now  at  most 
^0)  5oSi,  5oSiS2, ,  Sq  >  •  •  •  j  final  size  is  there¬ 

fore  bounded  by  . 

It  will  be  easy  to  see  that  actually  counts  the  number  of  steps,  if  s  is  the 
size  and  d  is  the  depth  :  we  are  therefore  polystep  in  s  (when  d  is  fixed,  which 
corresponds  to  practice).  Since  the  steps  are  not  to  big,  the  actual  runtime  is 
polynomial  in  the  number  of  steps,  and  the  complexity  of  cut-elimination,  for  a 
given  depth  d  will  therefore  be  poly  time. 

3.3.3  Elimination  of  exponential  cuts 
Definition  6 

The  actual  weight  of  a  discharged  formula  [A]  is  the  weight  of  the  conclu¬ 
sion  A  of  the  proof-net  inside  the  box.  An  exponential  cut  is  special  if  it  is 
a  ready  and  in  case  one  of  the  premises  of  the  cut  is  the  conclusion  of  a  ?- 
link,  then  this  link  is  either  0~ary  or  one  of  its  premises  has  actual  weight  1. 


We  now  explain  how  to  eliminate  special  cuts  :  this  is  the  special  procedure 

►  §-reduction  :  take  a  ready  cut  between  ^A  and  §A-‘-,  both  A  and  A-^  are  con¬ 
clusions  of  §-boxes  whose  contents  are  proof-nets  with  respective  conclusions 
r;A  and  A^]A  :  in  this  case  we  first  perform  a  cut  on  A  between  the  two 
proof-nets,  yielding  a  proof-net  with  conclusion  T;  A,  then  we  form  a  §-box 
with  this  proof-net. 

►  Weakening  reduction  :  take  a  special  cut  between  \A  and  ?A-^ ,  where  ?A-^  is 
the  conclusion  of  a  0-ary  link  :  in  this  case  we  remove  the  box  with  conclusion 
\A.  This  involves  the  destruction  of  the  conclusions  [5,]  of  this  box,  but  this 
only  amounts  to  reducing  the  arity  of  some  ?-links. 
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^  Contraction  reduction  :  take  a  special  cut  between  \A  and  where  is 
the  conclusion  of  a  ?-link  with  a  premise  [Ai-^]  of  actual  weight  1.  Then  [Ai^] 
is  in  turn  the  conclusion  of  a  box  B.  B  is  made  from  a  proof-net  S  whose 
conclusions  are  whereas  the  box  A  with  \A  among  its  conclusions 

is  made  from  a  proof-net  0  whose  conclusions  aieT;A.  By  means  of  a  cut 
between  A  and  Ai^^  we  can  produce  a  new  proof-net  11.  11  can  be  used  to 
produce  a  new  box  C  whose  conclusions  are  the  same  as  those  of  except 
that  [Ai-^]  is  replaced  with  [T],  In  this  case  we  replace  B  with  C.  Observe 
that  new  occurrences  of  [F]  are  created,  hence  the  arity  of  some  ?-link  will 
increase. 

What  about  the  size  during  this  procedure  ?  Let  us  assume  that  our  special  cut 
is  of  depth  0,  and  that  our  original  sizes  are  sq,  si,  • .  • ,  Sd  ; 

►  §-reduction  :  the  size  obviously  decreases  by  2,  since  three  links  (two  boxes 
and  a  cut)  counting  for  1  -1-  n  -b  1  +  m  are  replaced  with  two  links  (one  box 
and  a  cut),  counting  for  1  +  (n  -  1)  -|-  (m  -  1)  -f  1.  A  new  estimate  for  the 
partial  sizes  is  sq  “  2,  si , . . . ,  Sd  ; 

^  Weakening  reduction  :  the  size  strictly  shrinks,  and  sq  -  !>  , . . . ,  Sd  is  a  very 

pessimistic  majorization  of  the  size  of  the  result  ; 

►  Contraction  reduction  :  at  depth  0  the  size  stays  the  same,  since  C  has  the 

same  size  as  B  (this  is  because  there  is  no  semicolon  in  F,  so  that  F;  A  has  the 
same  number  of  semi-colons  as  A).  But  otherwise  it  increases  :  more 
precisely,  if  the  partial  sizes  of  the  proof-net  0  are  toAi>  •  •  -j^d-ij  then  the 
partial  sizes  of  our  new  proof-net  are  exactly  so  >  4-  ?  ^2  "b  >  •  •  •  >  ^d  +  ^d-i  • 

In  the  first  round  we  systematically  perform  the  basic  procedure  at  depth  1 
together  with  the  special  procedure  at  depth  0.  The  point  of  the  basic  proce¬ 
dure  is  that  it  induces  changes  of  weights  inside  the  boxes,  and  therefore  some 
conclusions  of  the  proof-nets  inside  a  box  receive  a  new  weight  0,  in  which  case 
some  conclusion  [A]  of  the  box  disappears.  This  does  not  affect  the  size  of  the 
box  (the  number  of  semicolons  stays  the  same)  and  since  such  a  conclusion  was 
the  premise  of  some  ?-link,  this  only  induces  a  change  of  arity  of  the  ?-link.  Of 
course  some  conclusion  of  box  may  get  the  actual  weight  1,  which  can  fire  a  con¬ 
traction  reduction,  etc.  By  the  way  no  basic  reduction  at  depth  0  can  be  fired 
during  the  first  round,  and  this  is  why  we  may  assume  that  they  have  been  done 
during  a  preliminary  round.  Later  on,  in  the  second  round,  no  basic  reduction 
at  depth  0  or  1  will  occur  etc. 


3.3.4  Bounding  the  sizes 

Bounding  the  size  essentially  amounts  to  considering  the  first  round.  We  there¬ 
fore  assume  that  the  basic  procedure  has  been  completed  at  depth  0.  We  also 
make  a  simplifying  hypothesis,  namely  that  no  non- trivial  weight  remains  at 
depth  0  :  this  will  be  the  case  when  we  normalize  proofs  of  lazy  sequents^  see 
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below 

We  introduce  a  precedence  relation  between  discharged  formulas  :  [A\  <i  [B] 
when  [B]  is  conclusion  of  a  !-box  B  and  the  other  conclusion  of  the  box  \A-^  is 
the  premise  of  an  exponential  cut  whose  other  premise  ?A  is  the  conclusion  of 
a  ?-link,  with  [A]  among  its  premises.  By  the  correctness  criterion,  the  transi¬ 
tive  closure  <  of  precedence  is  a  partial  order.  We  can  therefore  consider  the 
forest  T  of  finite  sequences  ([>io]j  •  •  •)  [^n])  of  discharged  formulas,  such  that 
[Ao]  is  minimal  w.r.t.  <  and  ([Aq]  <i  . . .  <i  [An]).  A  discharged  block  is  a 
set  of  discharged  formulas  [A]  which  occur  among  the  conclusion  of  some  ex¬ 
ponential  box  of  depth  0,  made  from  a  proof-net  with  conclusion  F,  and  such 
that  [A]  is  a  block  of  F.  A  coherent  subforest  in  is  a  subforest  //  of  .F  such 
that  whenever  two  sequences  [5],[A],[S"]  and  [5],[B],[5"]  belong  to  //,  then 
either  [A]  and  [B]  are  the  same  or  they  belong  to  distinct  discharged  blocks. 
Given  fi,  we  can  define  the  multiplicators  pi[B  for  any  box  B  with  discharged 
conclusions  to  be  the  number  of  sequences  in  fi  such  that  the  last  element  of  the 
sequence  is  a  conclusion  of  5  ;  if  is  a  §-box  whose  conclusions  are  all  of  the 
form  §A,  let  fi(B)  =  1.  The  potential  sizes  of  0  are  defined  as  follows  :  for  each 
depth  2  9^  0,  we  can  write  s*-  =  Y^sf,  where  B  varies  through  boxes  of  depth 
0  (sf  is  just  the  contribution  of  the  proof-net  inside  B  to  size).  We  define 

ss = ^0,  =  Ymsi . . . , 

Proposition  1 

Assume  that  0  reduces  to  11  during  the  first  round.  Then  the  potential 

sizes  of  n  are  not  greater  than  the  potential  sizes  of  0. 

Proof.  —  We  already  know  that  the  size  does  not  increase  at  depth  0  ;  let 
us  check  the  property  at  any  other  depth,  typically  depth  1,  and  in  the  only 
problematic  case,  namely  the  contraction  reduction.  We  start  with  boxes  A 
and  B  in  0  to  produce  a  box  C  which  replaces  B.  If  oi,  bi  are  the  respective 
contributions  of  A  and  B  to  the  1-size  of  0,  then  then  contribute  as 
Pe{A)ai  H-  pe{B)bi  to  the  potential  1-size  of  0,  whereas  in  11  the  boxes  A  and 
B  contribute  to  the  potential  size  as  uu{A)ai  +  vn{C){ai  +  6i).  But  since  C  is 
obtained  by  merging  B  with  a  copy  of  A,  it  is  easy  to  construct,  given  u  a  p 
such  that  p{B)  —  v{C)  and  p{A)  =  1/(0)  A  I'iA).  This  proves  the  claim.  □ 
By  the  way  observe  that  any  maximal  p  will  yield  p{A)  ^  1,  hence  the  potential 
sizes  easily  exceeds  the  sizes  ;  on  the  other  hand  observe  that  coherent  subforests 
are  not  too  big,  since  they  cannot  branch  at  all :  this  is  due  to  the  peculiarities  of 
the  !  boxes.  Moreover,  thanks  to  acyclicity,  the  same  discharged  formula  cannot 
occur  twice  in  the  same  branch  :  in  other  terms  p{A)  cannot  exceed  the  number 
of  roots  of  ^  which  is  bounded  by  the  number  of  discharged  blocks,  and  this 
number  is  in  turn  bounded  by  sq.  This  is  why  the  first  round  yields  the  bounds 
«0, . ♦ * , 

10.  If  non-trivial  weights  appear  at  depth  0,  we  can  apply  the  constructions  of  this  subsection 
to  the  part  of  the  proof-net  which  is  of  size  1 

11.  This  is  the  only  point  where  ELL  diverges  from  LLL  :  in  ELL  coherent  subforests  do 
branch  ! 
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3.3.5  Bounding  the  runtime 

We  show  below  that  the  runtime  is  of  degree  3  in  the  size,  which  will  yield 
poly  time  complexity  of  degree  2^+^  for  our  algorithm.  It  suffices  to  compute  the 
complexity  of  the  first  round  : 

►  The  number  sq  dominates  both  the  number  of  steps  of  the  preliminary  round 

and  the  number  of  special  steps  which  are  not  contraction  reductions  ;  the 
number  so^i  dominates  the  number  of  basic  steps  in  the  first  round.  The 
number  of  contraction  reductions  performed  during  the  first  round  is  smaller 
than  the  maximum  size  of  a  coherent  subforest  of  T y  and  is  therefore  less  than 
6o,  where  6o  is  the  number  of  discharged  blocks  of  depth  0,  which  is  turn  is 
bounded  by  Sq-  The  number  of  steps  during  the  first  round  is  therefore  easily 
bounded  by  (sq  +  ; 

►  However,  the  number  of  steps  is  not  the  runtime  :  some  steps,  typically 
contraction  reductions  involve  a  duplication  of  the  structure,  which  means 
that  each  step  can  cost  at  most  the  actual  size  of  the  proof-net.  We  already 
observed  that  the  actual  size  is  quadratic  in  the  size  (which  is  bounded  by 
sqs)  hence  we  arrive  at  a  total  of  (sos)^s^  for  the  first  round. 

Without  being  very  cautious,  we  can  bound  the  total  runtime  by  something  like 
,  which  is  enough  for  our  purpose. 

This  does  not  mean  that  the  algorithm  cannot  be  improved.  The  decomposition 
in  rounds  is  rather  artificial  etc.  But  we  are  not  looking  for  efficient  imple¬ 
mentation,  just  for  a  proof-system  which  is  intrinsically  polytime,  and  that’s 
it. 


3.3.6  Lazy  sequents 

A  formula  is  said  to  be  lazy  when  it  contains  neither  the  symbol  &:  nor  higher 
order  existential  quantification.  A  sequent  is  said  to  be  lazy  when  all  the  formulas 
occurring  in  it  are  lazy. 

Proposition  2 

Let  O  be  a  proof-net  without  non-exponential  ready  cut  of  depth  0,  and 
assume  that  0  has  a  ready  cut  at  depth  0.  Then  one  of  the  conclusions  of 
0  is  a  non-lazy  formula  of  weight  1, 

Proof.  —  since  eigenweights  can  only  be  used  at  a  given  depth,  some  eigen- 
weight  is  used  at  depth  0,  and  we  can  look  for  a  &:-link  L  such  that  the  empire  of 
its  conclusion  is  maximal  w.r.t.  any  valuation.  Then  the  downmost  conclusion 
below  this  link  cannot  be  the  premise  of  a  cut  (in  which  case  we  can  show,  as  in 
[2],  that  the  cut  would  be  ready),  hence  it  must  be  a  conclusion,  and  its  weight 
is  bigger  that  the  weight  of  T,  so  it  is  equal  to  1.  □ 

As  a  corollary,  after  the  preliminary  round,  a  proof-net  whose  conclusion  is  lazy 
has  no  non-trivial  weight  at  depth  0. 
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Proposition  3 

After  the  first  round,  the  proof  of  a  lazy  sequent  has  no  cut  of  depth  0. 

Proof.  —  assume  that  the  first  round  is  completed,  and  consider  the  forest 
;  if  there  is  still  a  cut  of  depth  0,  then  there  is  a  sequence  ([j4o],  [.4i])  in 
and  [Ao]  is  the  conclusion  of  a  box  B.  Since  a  conclusion  of  B  is  the  hereditary 
premise  of  an  exponential  cut  and  the  contraction  reduction  does  not  apply,  then 
this  conclusion  must  have  a  non-trivial  weight.  Now  the  proof-net  11 ,  which  is 
in  B  has  a  conclusion  with  a  non-trivial  weight,  and  since  the  basic  procedure 
has  been  completed  for  11,  there  is  a  conclusion  C  of  11  which  is  non-lazy  and  of 
weight  1.  This  conclusion  yields  a  conclusion  of  B  and  : 

►  either  the  conclusion  is  a  formula  §C  ;  since  this  formula  is  non-lazy,  it  must 
be  the  premise  of  a  cut. . .  But  the  § -reduction  would  apply,  a  contradiction  ; 

►  or  this  conclusion  is  a  formula  !C,  which  must  also  be  the  premise  of  a  cut. 
In  this  case,  observe  that  C  <i  Aq,  a  contradiction  ; 

►  or  this  conclusion  is  the  premise  [C]  of  a  ?-link,  which  must  in  turn  be  the 
premise  of  a  cut  ;  in  this  case  [C]  is  of  actual  weight  1,  and  the  contraction 
elimination  does  apply,  a  contradiction. 

Therefore  is  trivial  and  0  is  cut-free  at  depth  0.  □ 


Theorem  3 

Cut-elimination  converges  to  a  (unique)  normal  form  for  proofs  of  lazy 
sequents  ;  furthermore,  for  bounded  depth,  the  runtime  is  polynomial  in 
the  size  of  the  net. 

Proof.  —  more  or  less  obvious  from  what  precedes.  □ 

Observe  that  application  of  a  function  of  type  I*';  bint  i-  §*bint  to  an  argument 
falls  into  this  case  :  a  (cut-free)  argument  is  of  depth  1,  hence  the  global  depth 
is  the  depth  of  the  proof  representing  the  function.  Observe  that,  unlike  other 
approaches,  like  [7,  5],  there  is  still  a  complexity  bound  for  arbitrary  functionals, 
something  like  ,  since  the  size  clearly  exceeds  the  depth  by  2. 


A  APPENDIX 

A.l  Naive  Set-Theory  and  LLL 

We  have  so  far  only  considered  second  order  propositional  LLL.  But  this  is  not 
the  only  possibility  : 

►  We  can  consider  first-order  LLL,  which  is  straightforward. 

►  We  can  also  consider  LLL  with  first  and  second  order  quantifications  ;  this 
system  would  be  a  natural  candidate  for  a  light  second  order  arithmetic.  By 
the  way  a  light  first  order  arithmetic  could  easily  be  extracted,  but  one  would 
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have  to  think  twice  in  front  of  the  difficulties  inherent  to  equality,  especially 
in  terms  of  proof-nets  (e.g.  certain  formulas  like  0^1  will  be  equivalent  to 
T,  hence  the  case  of  T  has  first  to  be  fixed)  ; 

►  We  can  also  consider  quantifications  of  any  order. . . 

^  And  last  but  not  least,  we  can  consider  Naive  Set  Theory,  which  encompasses 
all  kinds  of  quantification. 

In  fact.  Naive  Set  Theory  has  been  the  starting  point  of  LLL  :  I  was  looking 
for  a  system  in  which  the  complexity  could  be  expressed  independently  of  the 
complexity  of  the  cut-formulas.  In  particular  it  would  also  work  for  naive  set- 
theory,  since  there  is  a  well-known  (non- terminating,  for  obvious  reasons)  cut- 
elimination  procedure  for  it  ;  by  the  way,  it  had  been  observed  long  ago  by 
Grishin  [4]  that,  in  the  absence  of  contraction,  cut-elimination  works  So  I 
decided  to  translate  Russell’s  paradox  into  linear  logic  with  exponentials.  Using 
fixpoint  facilities  (see  below)  one  can  produce  a  new  constant  A,  which  has  the 
rules  (unary  links  in  terms  of  proof-nets)  :  from  lA  deduce  from  lA-^  deduce 
A. 

►  There  is  a  first  possibility  for  deriving  a  paradox  (here  a  proof-net  with  no 
conclusion),  which  is  based  on  dereliction  :  the  proof- net  has  depth  1,  but  the 
process  of  cut-elimination  does  not  converge  at  depth  0,  since  the  normaliza¬ 
tion  of  a  cut  with  dereliction  between  ?A  and  lA-^-  involves  an  ’’opening”  of 
the  box  with  conclusion  ! A'^  :  the  contents  of  this  box  is  ”  poured”  into  depth 
0,  so  that  the  size  sq  no  longer  shrinks  ; 

^  There  is  another  possibility  which  does  not  use  dereliction,  but  the  principle 
??A-o?A  ;  in  this  case,  the  first  round  is  easily  completed,  but  the  handling 
of  the  exponential  cuts  involves  the  creation  of  a  deeper  box,  i.e.  the  size 
increases. 

This  is  why  we  restricted  to  rules  whose  normalization  involves  no  change  of 
depth. 

A.  1.1  Expressive  power  of  LLLs 

Light  Naive  Set-Theory  LLLs  is  defined  exactly  as  LLL,  but  for  the  quantifiers 
and  the  terms  : 

Definition  7 

Terms  (T)  and  formulas  (F )  are  defined  as  follows 
T  =  x,y,z,,.\  {x\F} 

=  r  G  17,  T  ^  U,  1 ,  X,  0,  T,  !F,  §F,  ?F,  F(g)F,  F^F,  FkF,  F^F,  VarF,  3xF 


12.  This  is  not  very  helpful,  since  the  system  without  exponentials  is  awfully  inexpressive  in 
tenns  of  computational  power 
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Negation  is  defined  as  expected  ;  in  particular,  {T  eU)-^  z=T  and 
{T  =T  eu. 

The  logical  rules  are  modified  as  follows  : 


i-T]A 

-  (for  all :  a  is  not 

i-r;Va:A  free  in  T) 


•-r;3x^ 


{there  25) 


»-r;A[r/a;] 

i^T^,Te{x\A} 

The  representation  in  terms  of  proof-nets  is  straightforward  :  the  €-rules  induce 
two  unary  links,  one  with  premise  A^T/x]  and  conclusion  T  e  {x  \  A},  the  other 
with  premise  {A[T/x])-^  and  conclusion  T  ^  {x  \  A}. 

A, 1.2  Equality 
Definition  8 

The  Leibniz  equality  t  =  u  is  defined  by  Vx(t  e  x -ou  e  x)  ;  t  ^  u  is  short 
fort  =  u-oO,a  strong  form  of  negation. 

Exercise,  —  Prove  the  following  sequents  : 

►  t  =  u\A[tlx]\- Alujx]) 

^  t  ~  ui-  u  —  t 

^  t  =  ut~{u=zv—0t  =  v) 

^  t  =  «  »-  1 

^  t  =  u^~  t  =  u^t  =  u 

Definition  9 

The  singleton  {<}  is  defined  as  {x  \  x  =  t]  ;  the  pair  {t,u}  is  defined  as 
{x\x  =  t^x  =  u}  ;  the  ordered  pair  <t,u>  is  defined  as  {{t},  {t,  u}}. 

Exercise.  —  Prove  the  following  sequents 

►  {*}  =  {t*}  *-t  —  t’ 

►  {t>  w}  =  {<', «'}  »-  (t  =  t'  ©  t  =  «')  ®  (u  =  e  u  =  «') 

►  {t,  u]  =  {/',  u']i-{t  =  t'<g>u  =  u')e{t  =  u'(^u=zt')et  =  u 

►  {t,«}  =  (t  = /' <S)  «  =  u')  ©  (t  =  0  «  =  t') 

►  {t}  =  {t\u'}t-t  =  t'  <s>t  =  u' 

►  <t,u  >=<t',u*  >i-t  —  t' = 

Exercise.  —  Prove  the  formula  {x  |  0}  ^  {i}  ;  conclude  that  we  can  find  terms 
to, .  -  - ,  tn, . . .  such  that  U  ^  tj  is  provable  for  i  ^  j. 

As  a  consequence  of  the  exercise,  it  is  possible  to  represent  certain  features  of 
the  usual  equalitarian  predicate  calculus  : 


■-r;.4[r/x]-^ 
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►  we  can  represent  a  n-ary  function  letter  /  by  assigning  to  it  a  specific  term  ti 
coming  from  the  previous  exercise  ;  fti ..  .in  will  be  represented  as 

<  >,  using  a  n  -f  1-ary  pairing  function.  It  follows  from  the 

previous  exercises  that  usual  equality  axioms  are  satisfied  together  with 
fii  ...tn  ^  gui  ...Um  (when  f,g  are  distinct)  and 

ft\  .  .  An  =  gui  ,  ,  .Un  -ot\  -  Ui  ^  =  Un 

►  we  can  also  represent  predicates  by  means  of  fixed  variables  (generic  con¬ 
stants)  and  by  means  of  the  pairing  function  :  pt\  ..  An  becomes 

{ti, ...  ,tn)  €  where  ar  is  a  variable  assigned  to  p 

►  as  a  consequence,  we  have  access  to  a  representation  of  binary  strings  :  for 
this  we  only  need  a  constant  e  and  two  unary  successors  So,Si.  Equality 
axioms,  as  well  as  inequalities  Sot  ^  S\u,Sot  e,Sit  /  e  are  provable,  as 
well  as  Sit  =  SiU  — o  i  =  u  for  z  =  1, 2. 

A. 1.3  Fixpoints 

In  order  to  formulate  the  fixpoint  property,  we  introduce  the  following  nota¬ 
tion  :  the  substitution  of  an  abstraction  term  Aiiri  . .  .Xn.B  for  a  n-ary  predicate 
symbol  P  in  the  formula  A  consists  in  replacing  any  atom  Pti . .  .tn  of  A  by 
B]^\/ Xi^  . . .  ^tnf Xf^. 

Proposition  4 

Let  A  be  a  formula  in  the  language  of  LLLs  augmented  by  means  of  a 
n-ary  predicate  P,  and  let  Xi,...,Xn  be  variables,  so  that  we  can  write 
our  formula  A[P,xi, . .  ;  then  there  is  a  formula  B  (depending  on 

^ly’^’jXn)  such  that  the  equivalence  (i.e.  both  linear  implications)  be¬ 
tween  A[Arci . . . Xn.B[xi, . . . ,  a?n]  and  B  is  provable. 

Proof.  —  This  is  a  straightforward  imitation  of  Russell’s  paradox  (already  used 
in  the  fixpoint  theorem  of  A-calculus).  For  instance,  let  us  assume  that  n  =  1  ; 
then  we  can  form  t  :=  {z  \  3x3y.z  =<  x,y  >  (8)A[Au;.  <  w,y  >€  y,x]}.  Then 
<  x,t  >£  t  is  provably  equivalent  with  A[Xw.  <  w,t  >£  t,x]  and  we  are  done. 

□ 

As  a  consequence  we  get  the  possibility  of  defining  various  partial  recursive  func¬ 
tions.  Typically,  take  for  instance  the  exponential  function  (defined  on  binary 
strings,  i.e.  in  number  base  2)  then  we  can  get  a  two  variable  formula  B  such 
that  B[s,t]  expresses  that  t  =  2\ 

This  is  enough  to  convince  one  that  LLLs  bears  all  the  features  of  a  light  arith¬ 
metic.  In  particular  all  numerical  functions  implicit  in  proofs  made  in  this  system 
will  be  polytime  computable. 


A. 2  Elementary  Linear  Logic 

Elementary  Linear  Logic  arise  as  the  alternative  solution  to  the  complexity  prob¬ 
lem  at  stake.  It  syntax  does  not  contain  §  (or  rather  does  not  need  it).  The  rule 
for  !  is  liberalized  into  : 
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(of  course) 


where  the  symbols  , . . . ,  j5„  are  separated  by  commas  or  semicolons. 

As  a  consequence,  the  sequent  \A;  \(A-o  B)  t-\B  becomes  provable  (equivalently 
\A]\B  i-!(A  (g)  jB)  is  provable).  Integers  can  now  be  represented  by  th6  type 
Vo.  \{a  -o  a)-o!(a  — o  a)  :  the  tally  integers  can  be  given  this  type,  which  was 
not  the  case  for  LLL.  The  representation  results  of  LLL  persist  (replace  §  by  ! 
everywhere).  We  can  also  get  rid  of  the  irritating  markers  1^  in  the  representa¬ 
tion  theorem,  since  the  rule  for  !  is  now  valid  with  an  empty  context.  But  new 
functions  arise,  namely  exponentials.  This  is  due  the  fact  that  multiplication 
can  now  be  given  the  type  int;  int  i-  int.  If  we  feed  the  first  argument  with  the 
integer  2,  we  can  type  duplication  with  int  int,  and  as  soon  as  duplication  can 
be  given  a  type  A  t-  A,  then  we  can  iterate  it,  yielding  a  representation  of  the 
exponential  function.  The  exponential  can  therefore  be  typed  with  int  i-!int, 
and  towers  of  exponentials  with  the  type  int  i-!^int.  The  same  holds  for  other 
data  types,  and  therefore  we  conclude  that  all  elementary  functions  (i.e.  func¬ 
tions  whose  runtime  is  bounded  by  a  tower  of  exponentials)  can  be  typed  in 
ELL. 

Is  this  optimal  ?  The  proof  of  normalization  still  holds  but  for  the  fact  that 
coherent  subforests  are  not  so  simple,  since  they  may  branch.  The  multiplication 
factor  involved  in  the  first  round  is  no  longer  sq  but  depends  exponentially  on 
So,  something  like  sj°.  Completing  the  process  will  therefore  cost  a  tower  of 
exponentials,  the  height  of  the  tower  depending  on  the  depth  of  the  proof-net. 
Hence  normalization  is  elementary  in  the  size  of  the  input,  when  the  depth  is 
given.  This  is  analogous  to  the  familiar  bounds  for  predicate  calculus/simply 
typed  A-calculus,  but  here  the  height  of  the  tower  does  not  depend  on  the  cut- 
formula,  but  on  more  hidden  parameter,  the  depth. 

It  is  also  possible  to  build  a  naive  set- theory  ELLs.  Its  expressive  power  is 
considerably  bigger  than  before,  since  the  exponential  function  plays  a  decisive 
role  in  mathematics.  This  induces  a  strange  system  which  can  both  formalize  a 
bunch  of  mathematics,  and  which  admits  definition  by  fixpoint.  Such  a  system 
seems  to  be  the  optimal  candidate  for  formalization  of  AI. 


A. 3  Questions 

Semantics 

What  is  the  natural  semantics  for  LLL  ?  We  can  of  course  take  the  usual  seman¬ 
tics  of  linear  logic,  e.g.  coherent  spaces,  but  we  shall  be  embarrassed  to  explain 
why  certain  principles  are  wrong.  This  question  is  presumably  the  deepest  con¬ 
nected  with  our  new  system  :  for  the  first  time  polynomial  time  appears  as  the 
result  of  the  free  application  of  logical  principles  which  are  in  no  means  contrived 


13.  Even  with  a  non-lazy  procedure  :  normalizing  non-ready  cuts  increases  the  size  by  an 
exponential  factor. 
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to  achieve  this  goal.  A  semantics  of  LLL  would  therefore  be  a  general  semantics 
of  polytime.  This  might  be  very  rewarding  :  remember  that  polytime  has  been 
characterized  in  many  ways,  but  always  through  presentations  «  A  function  is 
polytime  iff  it  can  be  obtained  by  means  of. . .  »,  and  nobody  knows  how  to  deal 
with  a  presentation.  On  the  other  hand  a  semantic  characterization  would  insist 
on  something  like  preservation  properties  etc.  that  a  mathematician  can  more 
easily  reason  about. 

The  connective  «  §  » 

This  strange  connective  has  been  introduced  to  compensate  two  things,  namely 
the  want  of  dereliction,  but  also  the  failure  of  the  principle  [V]  :  \A(S>\B  i-!(A(S)jB), 
which  is  essential  in  the  representation  of  data  types.  Surely  §A(S)§B  i-  §(A(2)5) 
holds  and  §(A  ©  B)  i-  §A  (g)  §J9)  fails,  but  there  are  principles  (typically  the  self¬ 
duality  of  the  connective)  that  have  been  added  on  the  sole  grounds  of  their 
simplifying  character.  Later  investigations  (in  particular  semantical  ones)  could 
help  to  clarify  this  question.  In  a  similar  way,  the  fact  that  !1  is  not  provable 
is  backed  by  good  taste  (!l  looks  like  the  0-ary  case  of  [V]),  but  by  no  deep 
intuition. 

Completeness 

In  some  sense  LLL  and  ELL  are  complete,  since  the  complexity  bounds  are 
here  once  for  all.  This  is  even  more  conspicuous  with  their  naive  set- theoretic 
extensions  :  what  could  be  more  powerful  than  unrestricted  comprehension  ?  In 
some  sense  the  theorems,  the  algorithms  coming  from  these  systems  should  be 
absolute.  Is  it  possible  to  make  sense  of  this  informal  remark  ? 

Execution 

In  our  systems,  the  runtime  is  known  in  advance,  depending  only  on  the  depth 
and  size.  We  could  seek  an  untyped  calculus,  with  a  notion  of  depth,  and  for 
each  depth  d  a  function  rd(.)  with  the  following  property  :  after  rd{^{t))  steps, 
then  we  reach  either  a  normal  form  or  a  deadlock.  There  should  be  two  solutions, 
corresponding  to  polytime  and  elementarity. 
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Intrinsic  Theories  and  Computational  Complexity 


Daniel  Leivant 
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Abstract.  We  introduce  a  new  proof  theoretic  approach  to  computa¬ 
tional  complexity.  With  each  free  algebra  A  we  associate  a  first  order 
“intrinsic  theory  for  A”,  IT(A),  with  no  initial  functions  other  than  the 
constructors  of  A,  and  no  axioms  for  them  other  than  the  generative 
and  inductive  axioms,  which  delineate  A.  The  case  most  relevant  to  tra¬ 
ditional  proof  theory  is  A  =  N  (the  unary  natural  numbers),  and  the  case 
most  relevant  to  computer  science  is  A  =  W  =  {0,1}*.  An  algorithm  is 
provable  if  it  provably  maps  inputs  in  A  to  values  in  A,  and  a  function 
is  provable  if  it  has  a  provable  algorithm.  We  show  that  the  provable 
functions  of  IT(N)  are  exactly  the  provably  recursive  functions  of  Peano 
Arithmetic. 

We  further  show  that  function  provability  is  equivalent  to  computational 
complexity  for  the  following  pairs  theory/complexity-class:  (1)  A  rami¬ 
fied  variant  RT(A)  of  IT  (A)  and  elementary  functions.  (2)  RT(W)  with 
quantifier-free  induction  and  poly- time;  (3)  RT(N)  with  quantifier-free 
induction  and  linear  space  (on  register  machines). 

Intrinsic  theories  combine  lean  axiomatics  with  expressive  flexibility, 
since  they  permit  explicit  (uncoded)  reference  to  arbitrary  computable 
functions.  Thus,  the  characterizations  above  provide  user-friendly  for¬ 
malisms  for  feasible  mathematics,  in  which  non-feasible  algorithms  can 
be  mentioned  freely.  Moreover,  natural  deduction  calculi  for  these  for¬ 
malisms  correspond  directly,  via  formula- as-type  homomorphisms,  to  ap¬ 
plicative  programs. 


1  Intrinsic  theories 

1.1  Intrinsic  theories  for  free  algebras 

Let  A  be  a  free  algebra,  generated  from  constructors  ci . . .  c^,  where  arity(ci)  ~ 
Ti  >  0.^  Let  arity{A)  =df  maxirj.  A  free  algebra  of  arity  1  is  a  word  algebra. 
For  example,  the  word  algebra  N  with  constructors  0  and  s  (of  arity  0  and  1 
respectively)  is  isomorphic  to  the  natural  numbers,  and  the  word  algebra  W,  with 

*  Work  partially  supported  by  NSF  grant  CCR-9309824 

^  That  is,  Ct  are  function  identifiers,  and  A  consists  of  the  closed  terms  generated  from 
them. 
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constructors  6,  0  and  1  (of  arity  0,1,  and  1  respectively)  is  essentially  {0, 1}*; 
e.g.  0(1  (1(€)))  can  be  identified  with  Oil. 

The  full  intrinsic  theory  for  A,  IT” (A),  is  a  theory  in  first  order  logic  with 
equality,  defined  as  follows.  The  vocabulary  consists  of  a  unary  relation  identifier 
A  and  the  constructors  of  A.^  For  a  list  t  =  ti  ...tj.  of  terms,  we  let  A(t) 
abbreviate  the  conjunction  A(ti)  A  •  •  •  A  A{tr).  The  axioms  of  IT(A)  are; 

-  Generative  axioms:  \fui  ...Un  A(u)  — ^  A(ci(u))  {i  =  1 . .  .k). 

-  Limitative  axioms:  All  instances  of  A-Induction,^ 

Vx  (A(a;)  ->  C1a[^u^^]-^^1^]u) 

where  C1a[^uM  ^df  Ai=i...k 

Clci  [Xu.if]  =df  V?;i . . .  Vn  ( (p[vi]  A  •  •  -  A  ^[v^]  9?[ci(v)] ) 

The  axioms  for  equality  are  Wx.x  =  x^  and  all  formulas  of  the  form 

'ix,y,  {x  —  y  A  (^[x]  ^  ^[y]),  where  (p  is  atomic.  Finally,  let  IT” (A)  denote  the 

fragment  of  IT^(A)  with  induction  restricted  to  existential  formulas.^ 


1.2  Provable  functions 

We  use  as  computation  model  Herbrand-Godel  style  equational  programs,  as 
defined  e.g.  in  [Lei94b],  and  we  refer  to  the  notion  of  coherent  program  defined 
there.  This  model  lends  itself  to  direct  rendition  in  intrinsic  theories,  without 
coding  or  auxiliary  concepts  or  notations,  and  without  invoking  formalisms  that 
allow  non- denoting  terms  (see  §6.2).  If  P  is  an  equational  program,  we  write  P 
for  the  conjunction  of  the  universal  closures  of  all  equations  in  P.  The  following 
observation  seems  to  be  part  of  the  folklore  (see  [Lei94b]  for  a  proof). 

Theorem  1.1  Let  (P,  f)  be  a  coherent  equational  program  over  A.  The  function 
f  over  A  computed  by  (P,  f)  is  total  iff  P  A(x)  A(f(x))  is  true  in  every 
model  of  P  in  which  the  denotation  of  A  equals  the  set  of  denotations  of  A-terms. 

This  equivalence  motivates  the  following  definition.  Let  T  be  a  theory  whose 
vocabulary  contains  that  of  IT'^(A).  We  say  that  a  function  /  over  a  free  algebra 
A  is  provable  in  T  iff  it  is  computed  by  some  equational  program  (P,  f)  over  A, 
such  that  T  h  P  A(x)  ->  A(f(x)).® 

^  We  make  an  analogous  typographical  convention  for  specific  algebras;  for  instance, 
W  is  used  for  the  algebra  W,  and  N  for  N. 

^  We  let  un-parenthesized  implications  associate  to  the  right.  Also,  we  write  E\t\u  for 
the  result  of  substituting  i  for  all  free  occurrences  of  variable  u  in  the  expression  E. 
We  omit  the  subscript  when  in  no  fear  of  confusion. 

®  That  is,  formulas  whose  canonical  prenex  form  is  purely  existential. 

®  We  use  boldface  f  as  formal  identifier  for  the  function  /,  and  similarly  for  other 
characters. 
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Of  course,  an  intrinsic  theory  can  be  assigned,  more  generally,  to  any  data 
system,  i.e.  to  a  sorted  structure  whose  universes  are  defined  by  simultaneous 
closure  conditions  (see  e.g.  [Lei90a]).  We  do  not  refer  here  to  such  generalizations. 

1.3  Classically  and  constructively  provable  functions 

It  is  well  known  that  the  same  functions  over  N  are  provably  recursive  in  first  or¬ 
der  arithmetic  regardless  of  whether  the  underlying  logic  is  classical,  constructive 
(intuitionistic),  or  minimal.^  The  proof  of  the  analogous  statement  for  provable 
functions  is  trivial: 

Theorem  1.2  If  a  function  over  A  is  provable  in  IT'^(A)  using  classical  logic, 
then  it  is  provable  already  using  minimal  logic. 

Proof.  Assume  that  P  -*>  A(x)  — >  A(f(x))  is  provable  in  IT^(A)  based  on 
classical  logic.  By  standard  double-negation  translations  (see  e.g.  [Lei85]),  this 
implies  that  P  — >  A(x)  — ►  -i“iA(f(x))  is  provable  using  minimal  logic,  i.e.  P 
A(x)  — ^  ((A(f(x))— ^±)”>_L)  is  provable.  Since  minimal  logic  has  no  rule  for  _L, 
the  latter  formula  remains  provable  in  minimal  logic  after  (correct)  substitution 
of  any  formula  for  _L.  Substituting  A(f(x))  yields  P  — ^  A{x)  — ^  A(f(x)).  □ 

1.4  Equality- free  intrinsic  theories 

Let  P  be  an  equational  program  over  A.  Say  that  terms  t  and  t'  are  P-equivalent 
if  there  is  an  equation  q  =  s  in  P,  and  a  substitution  a,  such  that  t^  is  obtained 
from  t  by  replacing  an  occurrence  of  aq  by  as,  or  vice  versa.  Let  IT(A)  be 
the  fragment  of  IT“(A)  without  equality,  and  let  IT(A,  P)  be  IT(A)  with,  as 
additional  axioms,  all  formulas  A(t)  A(t'),  where  t'  is  P-equivalent  to  t. 
ITo(A)  and  ITo(A,P)  are  the  corresponding  theories  with  induction  restricted 
to  existential  formulas.  We  say  that  (P,  f)  is  provable  in  IT(A)  if  IT(A,  P)  h 
A(x)-^(f(x)). 

Proposition  1.3  An  equational  program  (P,  f)  is  provable  in  IT“(A)  iff  it  is 
provable  in  IT(A).  Similarly  for  IT^(A)  and  ITo(A). 

Proof.  If  (P,  f)  is  not  provable  in  IT(A)  then,  by  completeness,  S  ^  A(x) 
A(f(x))  for  some  model  S  of  IT(A,  P).  Let  be  S  augmented  with  the  in¬ 
terpretation  of  equality  as  {{a,h)  ]  a  =  6  is  derived  from  P  using  names  for  all 
elements  of  S}.  Then  S~  is  a  model  of  IT~(A)  P  but  not  of  A(x)  ^  A(f(x)), 
so  (P,  f)  is  not  provable  in  IT^(A).  The  proposition’s  forward  implication  holds 
also  for  constructive  and  for  minimal  logic. ^  By  Theorem  1.2.  The  backward 
implication  is  trivial.  □ 

We  use  a  natural  deduction  calculus  for  IT(A,P),  where 

^  The  simplest  proof  of  this  fact  is  due  to  H.  Friedman  [Fri78].  Minimal  logic,  also 
called  positive  logic,  is  the  fragment  of  constructive  logic  with  no  rules  for  negation 
or  falsehood. 

®  A  proof  theoretic  argument,  not  given  here,  yields  low  complexity  bounds  on  the 
increase  of  proof  size. 
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—  The  generative  axioms  of  IT(A)  are  formulated  as  inference  rules:  from  the 
ri  premises  A(ti), . . .  A{tri )  (where  ti . . . ,  t^.,.  are  terms)  infer 

A(ci(ti 

-  The  A-induction  principle  is  formulated  as  an  inference  rule:  From  the  Aj+I 
premises  A(t)  and  Clal^u.ip]  {i  =  l...k)  infer 

~  An  inference  rule  [P]:  from  a  formula  A(t)  infer  A(t'),  provided  t'  is  P- 
equivalent  to  t.  By  Proposition  1.2  we  may  assume,  without  loss  of  generality, 
that  implication  is  the  only  propositional  connective  used,  and  V  the  only 
quantifier.^®  Alternatively,  one  may  easily  expand  the  development  below  to 
apply  to  the  remaining  logical  constants. 


1.5  The  provable  functions  of  Peano  Arithmetic 

Recall  that  a  function  /  :  N  N  is  provably  recursive  in  Peano  Arithmetic  PA 
if  f(x)  =  py.fp  where  is  a  formula  of  PA  with  all  quantifiers  bounded,  with 
x^y  as  the  only  free  variables,  and  such  that  PA  h  VxBj/.  (p.  In  this  section 
we  show  that  every  provably  recursive  function  of  PA  is  a  provable  function  of 
IT(N). 

Let  FA  be  PA  with  function  identifiers  for  all  primitive  recursive  functions, 
and  with  the  defining  equations  for  them.^^  Let  FA“  be  FA  without  Peano’s 
third  and  fourth  axioms.  The  following  is  proved  in  [Lei91]. 

Lemma  1.4  If  f  is  a  provably  recursive  function  of  PA  then  there  is  a  primitive 
recursive  function  g  such  that  f{x)  =  fxy.  i9{x,y)=0)  and 
FA~  h  ^x3y  g(x,  y)  —  0. 

A  straightforward  induction  on  primitive  recursive  function  definitions  estab¬ 
lishes  the  following 

Lemma  1.5  Every  primitive  recursive  function  is  provable  in  IT(N). 


Lemma  1,6  If  a  formula  (p  is  provable  in  FA“,  by  a  proof  theat  refers  to  func¬ 
tions  then  ip^  (that  is,  cp  with  all  quantifiers  relativized  to  N)  is 

provable  in  IT^(N),  from  the  dehning  equations  for  /i . . .  /m- 

Proof.  Straightforward  induction  on  proofs.  Lemma  1.5  is  used  for  the  quantifier 
rules.  □ 

Theorem  1.7  Every  provably  recursive  function  of  PA  is  provable  in  IT~(N). 

®  Note  that  this  is  an  inference  rule  in  the  sense  of  natural  deduction,  and  not  the 
Induction  Proof  Rule  (cf.  e.g.  [Par77]);  its  restriction  to  a  class  ^  of  formulas  is 
therefore  equivalent  to  the  Induction  Schema  restricted  to 
In  this  context  “existential  formula”  should  be  read  “quantifier  free  formula.” 

For  all  applications  it  suffices  to  take  Kalmar-elementary  functions. 
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Proof.  Suppose  that  /  is  a  provably  recursive  function  of  PA.  By  Lemma  1.4 

FA“  f- Va;32/ g(a:,  y)  =  0  (1) 

where  p  is  as  above.  By  Lemma  1.6  it  follows  that 

IT={N),Poi-'^x^3y‘^e{x,y)  =  0  (2) 

where  Pq  consists  of  the  defining  equations  for  the  functions  used  in  the  proof 
for  (1). 

Without  loss  of  generality,  we  may  assume  that^^ 

Va;3!|/g(a:,2/)  =  0  (3) 

Let  P  be  Pq  augmented  by  the  equations 

HO,y)  =  y  and  f{x)  =  h{s{x,y),y)  (4) 

(where  h  and  f  are  fresh).  Then  P  is  coherent  by  (3),  and  /  is  computed  by 
(Pjf),  i.e.  f{n)  =  m  iff  f(n)  =  m  is  derivable.^®  (Note  that  h(0,m)  =  m  is  an 
instance  of  P,  and  g(n,m)  =  0  is  derivable  from  Pq.)  By  (2)  and  (4)  it  follows 
that 

IT=(N),  P  h  N{x)  N{f{x)). 

□ 


2  Provability  and  applicative  programs 
2.1  Recurrence  over  A 

It  is  well  known  that  natural  deductions  of  suitable  calculi  can  be  viewed  as  A- 
terms.^^  In  [Lei83,  LeiQOa]  we  modified  this  mapping  to  a  homomorphism,  which 
yields  directly  an  applicative  program  for  a  function  /  from  a  derivation  for  the 
provability  of  /  in  various  formalisms.  This  section  is  an  expanded  revision  of 
[Lei90a,  §5]. 

Let  lA  be  the  typed  lambda  calculus.  We  let  associate  to  the  right,  and 

we  write  pj, . . . , r  for  ► - >  r;  if  all  p^’s  are  the  same  type  p, 

we  write  p^  for  the  above.  Let  Rec(A)  be  lA  augmented  with  the  following 
constants:  (1)  for  each  constructor  c^  of  A,  a  constant  Ci  of  type  (Ti[o],  where 
(^i[p]  =df  P  (recall  that  u  =  arity{ci))’,  (2)  for  each  type  r,  a  constant 

R^J'  =  Hr  of  type  o,  o-i  [r], . . . ,  o-fe[r]  — ►r.  The  reduction  rules  of  Rec(A)  are:  the 
/3-rule,  and  for  each  type  r  the  rule  of  recurrence  in  type  r: 

Rr(ciai ..  MiAi^'-An  where  Aj  =df  •’*  Afjb 

Otherwise  replace  g  by  g'{x,y)  —dt  if  'iz<y.g{x,z)j^0  then  gix^y)  else  H-p(rc,2/). 
n  =  the  n’th  numeral. 

This  is  the  Schonfinkel-Howard  isomorphism,  also  known  as  the  Curry-Howard 
formula-as-type  analogy  [Sch24,  HowSO]. 
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We  write  ReCo(A)  for  the  fragment  of  Rec(A)  with  Rr  used  only  for  r  =  o. 

A  term  is  normal  if  no  subterm  can  be  reduced.  The  familiar  Tait-Prawitz 
method  [Pra71]  is  easily  applicable  to  Rec(A),  yielding 

Lemma  2.1  Every  reduction  sequence  in  Rec(A)  terminates. 

Note  that  each  element  of  A  is  represented  in  Rec(A)  by  itself  (modulo 
currying),  with  type  o,  and  that  every  closed  normal  terms  of  type  o  is  an 
element  of  A.  Thus,  every  expression  of  type  represents  an  r-ary  function 

over  A. 

Clearly,  the  constant  R^  denotes  the  operation  of  iteration  in  type  r,  that  is 
the  function  (gi  where  /  is  defined  by  f(ci(ai  ...an))  = 

9i(f(^i)  •  •  •  f(^ri))‘  The  following  observation  is  therefore  trivial. 

Proposition  2.2  Every  function  over  N  represented  in  Rec(N)  is  generated  by 
iteration  in  Unite  type  (and  explicit  deSnitions). 

2.2  Proofs  as  applicative  programs 

The  natural  deduction  calculus  for  IT(A,  P)  (where  P  is  any  coherent  program) 
can  be  mapped  homomorphically  to  Rec(A),  as  follows.  Define  inductively  a 
mapping  k  from  formulas  to  types  by:  «(A(t))  =df  o;  k{iI)^x)  =cif  («V'-^«X); 
/c(Va;.'0)  =df  kV'.  Define  further  a  mapping  from  derivations  of  IT(A,  P)  to  terms 
of  Rec(A),  which  without  danger  of  ambiguity  we  ^so  denote  by  k.  If  7T  is  a 
derivation  from  labeled  open  assumptions  j  •  •  •  >  fo  conclusion  (^,  then  kU 
will  be  a  term  of  type  with  free  variables  of  types  K'tpi . . . 

respectively.^®  The  term  kU  is  defined  by  recurrence  on  il,  as  follows,  using 
the  convention  that  if  77  is  a  derivation,  then  77i,772,...  are  TT’s  immediate 
sub-derivations,  in  that  order. 

1.  If  77  is  a  labeled  open  assumption  then  k77  =  (the  j-th  variable  of 
type  Ki)). 

2.  If  77  derives  by  implication  introduction,  closing  labeled  assumption 

-  ,  then  /c77  =  Xxj'^ .  nil i . 

3.  If  77  derives  ip  by  implication  elimination,  from  ^  and  then  nil  — 
(/c77i)(/c772). 

4.  If  77  derives  ip  by  universal  introduction,  universal  elimination,  or  [P],  then 
nil  —  k77i. 

5.  If  77  derives  A(ci(ti . .  .t^J)  by  the  generative  rule  for  Ci,  then  nil  — 
Ci(K77i)---(/c77rJ. 

6.  If  77  derives  by  A-induction  from  i4(t)  and  ClcAXu.ip]  (i  =  l...k), 
then  /c77  =  Rkv7(«^i)(^-^2)  •  •  *  («-fffc+i)- 


We  stipulate  a  concrete  syntax  for  natural  deductions,  where  those  open  assumptions 
Ip  that  are  closed  jointly  at  some  inference  are  labeled  by  a  common  natural  number, 
not  used  as  a  label  elsewhere  in  the  proof. 
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2.3  Provability  equals  definition  by  recurrence 
The  following  three  lemmas  are  straightforward. 


Lemma  2.3  For  every  a  G  A  there  is  a  normal  deduction  Ga  of  IT( A),  deriving 
A{a),  such  that  nOa  =  a. 


Lemma  2.4  Jf  d  =  kII  is  a  closed  normal  term  and  U  derives  in  IT(A,P)  a 
formula  of  the  form  A(t),  then  d  G  A  and  P  h  t  =  d. 


Lemma  2.5  Suppose  U  is  a  deduction  deriving  tp  from  assumptions  If 

kII  reduces  to  E  in  Rec(A),  then  E  =  kU'  for  some  deduction  il',  also  deriving 
(p  from  . . .  ipg. 


Theorem  2.6  Suppose  that  the  equational  program  (P,  f)  computes  a  function 
f  over  A.  If  77  is  a  deduction  in  IT(A,  P)  deriving  A(x)  ^  A(f (x)),  then  kU 
represents  f  in  Rec(A). 

Proof.  Without  loss  of  generality,  let  /  be  unary.  Given  o  G  A  let  17 a  be  the 
result  of  substituting  a  for  free  occurrences  of  a:  in  77.  We  have  nlla  =  «77 
trivially.  Combining  11  a  and  Ga  by  implication  elimination  yields  a  proof  Ea 
of  A(f(a)),  with  KEa  =  («77a)(«0a),  which  by  Lemma  2.3  is  =  (/c77)(a).  By 
Lemma  2.1  KEa  reduces  to  a  closed  normal  term,  which  by  Lemma  2.5  is 
for  some  proof  E*^  of  A(fa).  By  Lemma  2.4  P  |=  /cX'Jj  =  /(a),  so  k77  represents 
/.  □ 


Theorem  2.7  A  function  f  over  A  is  provable  in  IT(A)  iff  it  is  definable  in 
Rec(A).  Similarly  f  is  provable  in  ITo(A)  iff  it  is  definable  in  ReCo(A). 

Proof.  The  forward  implication  follows  from  Theorem  2.6.  The  proof  of  the 
backward  implication  is  straightforward  (and  not  used  in  the  sequel).  □ 


Theorem  2.8  [God58]  The  provably  recursive  functions  of  PA  are  precisely  the 
functions  definable  by  iteration  in  all  hnite  types. 

Proof.  We  prove  the  forward  implication.  If  /  is  provably  recursive  in  PA,  then 
it  is  provable  in  IT“(N),  by  Theorem  1,7.  From  Proposition  1,3  it  follows  that 
/  is  provable  in  IT(N),  and  therefore  is  definable  in  Rec(N),  by  Theorem  2.7. 
Thus  /  is  definable  by  recurrence  in  finite  types.  □ 
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3  Ramified  intrinsic  theories 

3.1  Ramified  algebras 

In  this  section  we  introduce  ramified  variants  of  intrinsic  theories,  and  char¬ 
acterize  in  terms  of  ramified  provability  the  Kalmar-elementary  functions,  the 
poly- time  functions,  and  the  linear-space  functions.  The  conceptual  rationale  for 
ramification  is  discussed  in  §6.4  below. 

Given  a  free  algebra  A,  we  define  the  ramified  intrinsic  theory  for  A,  RT(A), 
based  on  first  order  logic  without  equality,  and  using  the  logical  constants  A, 
V  and  (for  classical  and  constructive  logic)  ±.  The  vocabulary  consists  of  the 
constructors  of  A  and  a  countable  list  of  unary  relation  identifiers,  Aq,  Ai  . . .. 
The  mathematical  principles  are  formulated  as  inference  rules,  like  for  IT  (A): 

-  Generative  rules:  for  each  ^  >  0,  from  A^(ti), . . . ,  infer  A^(ci(t)) 

“  Ramihed  A-induction:  For  each  ^  >  0  and  formula  ip  with  no  Aj  with  j  > 
infer  from  the  k+l  premises  A/(t)  and  Cl al^u. ip]  {i  =  l...k)  the  formula 
¥>[*]. 

-  A-selection:  Infer  from  the  A:+l  premises  Ao(t)  and  CIq.  [Xu.(p]  {i  ~  l..,k) 
the  formula  (/?[t],  where 

=df  Vui  ...Vn  Ao(v)  </?[ci(v)]. 

Note  that  the  un-ramified  form  of  the  Selection  schema  is  proved  by  A-Induction 
for  Xu.  A{u)  A  (p[u].  However,  the  ramified  Selection  schema  is  not  a  special  case 
of  ramified  A-Induction,  because  ip  may  refer  to  At  for  any  L  We  write  RTo(A) 
for  RT(A)  with  ramified  induction  restricted  to  formulas  without —►or 


3.2  Provable  functions  of  ramified  theories 

Let  r  be  a  theory  whose  vocabulary  contains  that  of  RT(A).  We  say  that 
a  function  /  over  a  free  algebra  A  is  provable  in  T  iff  it  is  computed  by  some 
equational  program  (P,  f)  over  A,  such  that,  for  some  j, 

rt-P^^,(x)^Ao(f(x)). 

A  trivial  ramified  induction  establishes  the  following: 

Lemma  3.1  lfm>i>0  then  RT(A)  h  Am{x)-^At{x). 

From  this  it  follows  that  (P,f)  is  provable  in  RT(A)  iff  RT(A)  h  P  — ► 
Aj^{xi)^ - ^i(f(x))  for  some  ji . 


If  3  and  V  are  present,  they  are  permitted;  thus  induction  is  restricted  to  positive 
existential  formulas. 


185 


Proposition  3.2  The  provable  functions  of  RT(A)  and  ofRTo(A)  are  closed 
under  composition. 

Proof.  Consider,  without  loss  of  generality,  the  composition  f(x)  =  h2(x)), 

where  g,  hi  and  /12  are  provable,  that  is  RT(A)  proves  P  — >  Aj(x,  y)  — ^  Ao(g(a:,  y)) 
and  P  Ae^{x)  Ao{hq{x))  {q  =  1,2)  for  some  where  P  is  a  program 

defining  /  via  g  and  h  as  above.  Incrementing  by  j  all  the  subscripts  of  A  in  the 
proofs  of  the  latter  formulas  yields  proofs  of  P  — >  A£^^j{x)  — »>  Aj(hg(x))  (q  = 
1,2).  Let  p  =  max(^i,^2)  +  j-  Then,  by  Lemma  3.1,  P  Ap{x)  Aj(hq(a;)) 
{q  =  1,2),  and  so  P  — ►  Ap{x)  — ►  Ao(f(a;)).  O 

We  say  that  a  program  (P,f)  is  hatly  provable  ii  T  1-  P  Ao(x)  — ►  Ao(f(x)). 
Analogously  to  Theorem  1.2  we  have: 

Theorem  3.3  If  a  function  over  A  is  provable  (Satly  provable)  in  RT(A)  using 
classical  logic,  then  it  is  provable  (respectively,  Satly  provable)  already  using 
minimal  logic. 


3.3  Examples  of  provable  functions 

Let  us  start  with  two  fiatly  provable  functions.  Consider  a  free  algebra  A  as 
above,  with  r  =  arity(A).  The  destructor  functions  for  A  are  the  r  functions 
defined  by 

dstrj(ci(ai  . . .  a^J)  =  if  1  <  j  <  U  then  Ui  else  Ci(ai  ...an)  j  =  l...r. 

The  case  function  for  A  is  the  {k  -\-  l)-ary  function  case  =  case^.  defined  by 
case(ci(a.),x^  .  ..Xk)  =  Xi. 

Lemma  3.4  The  destructor  functions  and  the  case  function  are  hatly  provable 
in  RTo(A). 

Proof.  Let  'ip[u]  =df  An( dstr  j(u)).  For  each  we  have  Ao(vi . . .  Vn)^'ip[ci(v)], 
by  the  definition  of  dstr^.  Thus  Ao(rc)  — ►  ^[x]  by  Selection,  so  dstr^  is  flatly 
provable.  The  proof  for  case  is  similar.  □ 

Two  simple  examples  of  provable  (but  not  flatly  provable)  functions  are  addi¬ 
tion  and  multiplication.  We  phrase  them  as  generic  functions  over  word  algebras 
A:i7 

0(ci,x)=x  (8)(ci,x)  =  Ci  ifri  =  0 

©(ci(a),  x)  =  Ci(©(a,  x))  (S>(ci(a),  x)  =  0(x,  (g»(a,  x)))  if  n  =  1 

In  the  sequel  we  use  ©  and  <S)  in  infix  whenever  convenient.  We  are  mostly 
interested  in  the  fact  that  Ingth  (a  (8)  6)  =  Ingth  (a)  •  lngth{b). 


A  yet  more  generic  multiplicative  function  could  be  defined  by  ©(c*, xi . .  .Xr)  =  Xi, 
0(ci(a),xi  ...Xr)  =  ©(xi,©(a,x)). 
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Lemma  3.5  Let  A  be  a  word  algebra.  The  functions  0  and  <S>  are  provable  in 
RTo(A).  In  fact,  for  all  £  >  0  the  formulas  A^+i(2/)  A  Ai{x)  ->  Ai{y  0  x)  and 
Ai^i{z)  A  At.^i{y)-^Ai[z  0  y)  are  provable  in  RTo(A). 

Proof.  Let  P  be  the  defining  equations  above  for  0  and  0.  Let  <p[u]  =df 
Ai{u  0  a;).  Under  the  assumption  Ai{x),  the  equations  P  imply  Cici[Xu.(p].  So 
by  ramified  induction 


A  At{x)  Ai{y  0  x)  (5) 

Now  let  il)[v]  =df  Aliy  0  y).  Then,  under  the  assumption  Ai^i{y)  and  P  we 
have,  by  (5),  ClalXv.ip].  So  by  ramified  induction  on  -0  we  obtain 

Ai+i{y)  A  Ai+i(z)  Ai{z  0  y). 


□ 

Prom  Lemmas  3.5  and  3.2  we  conclude 

Lemma  3.6  All  polynomial  functions  over  A  (i.e.  compositions  of  0  and  0)  are 
provable  in  RTo(A). 

Using  induction  for  universal  formulas  we  can  further  expand  the  collection 
of  provable  functions: 


Lemma  3.7  Numeric  exponentiation  is  provable  in  RT(N). 

Proof.  Let  P  be  the  program  with  principal  identifier  exp ,  and  consisting  of  the 
three  equations  e(a:,0)  =  sa:,  e(x,sy)  =  e(e(x,y),y),  and~exp(v)  =  e(0,v).  Then, 
by  induction  on  y,  e(x,y)  =  x  -h  2^,  so  P  computes  the  exponential  function 
A7i.2^. 

Let  ip[v\  =df  Vv  Aro(v)  — ►iVo(e(v, w)).  We  show  <^[w]  — ^<^[sw].  Assume  ^[u], 
and  consider  w.  From  (p[u]  we  get  No{w)^NQ{e{w,u))  as  well  as  No{e{w,u))^ 
NQ{e{e{w,u),u))).  Combining  these  implications  and  using  the  defining  equa¬ 
tions  for  e,  we  get  ^  iVo(e(it;,sii)),  from  which  (p[su]  follows  by  V- 

introduction.  Thus  (p\u\  — ►  (p[su].  Also,  y7[0]  is  immediate  from  the  genera¬ 
tive  axioms  for  Nq.  It  follows  by  induction  that  Ni{x)  implies  (p[x],  in  par¬ 
ticular  No{0)  — >  iVo(e(0,a:)),  and  therefore  Aro(e(0,a;)),  i.e.  No{exp{x)).  Thus 
Ni{x)^No(^{x)).  □ 

Define  the  functions  2k  (k  >  0)  by  2o(a;)  =df  x,  and  2jb+i(x)  =df  22*'^®).  From 
Lemmas  3.2  and  3.7  we  conclude 


Proposition  3.8  All  functions  2k  are  provable  in  RT(N). 


□ 
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4  Ramified  provability  from  complexity  classes 

4.1  Flat  provability  of  machine  transition  functions 

We  relate  provability  to  computation  in  a  natural  generic  model  of  register  ma¬ 
chines  (RMs)  over  free  algebras,  described  in  [Lei94c],  to  which  we  refer  the 
reader  for  detail.  A  RM  M  consists  of  a  set  S  =  {si . . .  of  states,  (si  and  si 
are  the  initial  and  terminal  state,  respectively),  a  list  iT  =  tti  ...  TTm  of  registers, 
and  a  finite  set  of  commands,  each  being  a  constructor  command,  a  destructor 
command,  or  a  branching  command.^®  We  posit  some  code  #Si  e  A  for  each 
state  Si. 


Lemma  4.1  Given  a  deterministic  RM  M,  there  are  m-Hl  (m-l-l)-ar7  func¬ 
tions  ro,ri, . . .  ,Tyn  that  are  ffatly  provable  in  RTo(A),  and  such  that  configu¬ 
ration  [s,  tti , . . . ,  Um]  has  an  M -transition  to  configuration  u'^]  iff 

Ti{i^s,Ui,..'.,Um)=u\  for  i  =  and  ro(#s,'ai, ...  ,u^)  =  #s'.  More¬ 

over,  if  M  has  no  transition  for  state  s,  then  Ti(uo  . . . Um)  =  Ui  (i  =  0 . .  .m). 

Proof.  Each  Uj  (j  =  is  either  uj,  or  is  obtained  by  applying  a  constructor 

or  a  destructor  of  A  to  some  iii’s.  Moreover,  the  appropriate  case  for  the  u'/s 
and  for  the  state-code  argument  #s'  can  all  be  determined  from  the  input  using 
the  function  case.  All  these  functions  are  flatly  provable  in  RTo(A)  by  Lemma 
3.4,  so  To  ...  are  also  flatly  provable  in  RTo(A).  □ 

4.2  Provability  of  functions 

Theorem  4.2  Let  A  be  a  word  algebra.  If  a  function  f  over  A  is  computable 
on  a  RM  over  A  in  polynomial  time  then  f  is  provable  in  RTo(A). 

Proof.  We  give  the  proof  for  A  =  W;  the  general  case  is  similar.  Assume  there 
is  an  m  such  that  f(w)  is  computed  in  <  |t(;p  steps  for  all  w  eW.  Write  r  for 
the  vector  of  functions  (ro-.-Tm)}  and  let  a  ==  (#si,tn,€, . . . e)  be  the  initial 
configuration  of  M.  By  Lemma  3.6  there  is  a  program  (Pm,T)  for  Xn.n^  such 
that 

RT,(W)  H  (6) 

for  a  suitable  £  >  1.  Let  P  consist  of  the  defining  equations  of  the  functions  Tj, 
the  equations  in  Pm,  and  the  following: 

cfy^(e,x)  =  the  j’th  entry  of  a 

x)  =  Tj(^o(<,  x),...  x))  c  =  0, 1 

{{x)  =  cfg^{T{x),x) 

A  constructor  commands  states,  roughly,  store  in  a  certain  register  the  result  of 
applying  Ci  to  the  current  contents  of  certain  registers;  a  destructor  command  is 
analogous;  and  a  branching  command  states  choose  the  next  state  among  a  given 
list  ofk  states,  according  to  the  main  constructor  of  the  contents  of  a  certain  register. 
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Then  (P,  f)  computes  /  (we  stipulates  that  teh  output  of  a  RM  is  read  off  the 
first  register). 

To  see  that  (P,f)  is  provable  in  RTo(W),  let 
^[t]  =df 

By  Lemma  4.1  and  P  we  have  RT(W)  h  Ciw[AL^].  Thus,  by  induction  bn 

RTo(W)  h  Wiit)  ^  ip[t]. 

Combining  this  with  (6)  we  get 

RT,{W,P)  h  We{x)  ^  Wo{^^{T{x\x)), 

that  is,  (P,  f)  is  provable.  □ 

Remark:  i  above  can  be  taken  to  be  A;  +  l.  In  [Lei94c]  a  different  applicative 
program  (P',  f)  for  /  is  defined,  for  which  only  two  levels  suflSce:  RTo(W,  P')  h 
Wi{w)~^Woif{w)). 

Prom  Theorem  4.2  we  immediately  conclude: 

Corollary  4.3  Every  function  Taring-computable  in  polynomial  time  is  prov¬ 
able  in  RTo(W)  using  quantifier-free  induction. 

The  same  holds  for  all  word  algebras  with  2  or  more  unary  constructors. 
However,  for  word  algebras  with  only  one  unary  constructor,  like  N,  the  relation 
between  RM  computation  and  Turing  computation  is  slightly  less  direct,  as  we 
have  the  following  (essentially  due  to  Hartmanis  and  Gurevich;  see  [Lei94c]  for 
a  proof): 

Lemma  4.4  A  numeric  function  f  is  computable  on  a  RM  over  N  in  polynomial 
time  iff  it  is  computable  on  a  multi-tape  TM  in  linear  space. 

Therefore,  Theorem  4.2  implies  in  this  case: 

Corollary  4.5  Every  numeric  function  computable  on  a  multi-tape  TM  in 
linear  space  is  provable  in  RT(N)  using  quantifier-free  induction. 

Using  in  the  proof  of  4.2  Lemma  3.8  in  place  of  Lemma  3.6  we  obtain: 

Proposition  4.6  Every  function  f  over  a  word  algebra  A  computable  in  Kalmar¬ 
elementary  time  is  provable  in  RT(A). 

Note  that  induction  is  used  here  for  universal  formulas.  Prom  Theorem  5.2 
below  it  follows  that  no  additional  functions  are  provable  when  induction  is  used 
for  formulas  of  arbitrary  quantifier  complexity. 

However,  if  only  the  first  two  ramification  levels  are  permitted,  then  quantifier 
complexity  of  induction  corresponds  to  iteration  of  exponentials,  as  we  shall  show 
elsewhere. 
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5  Complexity  classes  from  ramified  provability 

5.1  Ramified  proofs  as  ramified  programs 

Let  lA^  be  the  typed  lambda  calculus  with  pairing.  That  is,  given  a  set  of  base 
types,  the  types  are  generated  by:  if  ri  and  r2  are  types  then  so  are  ti  T2 
and  Ti  X  r2.  Given  a  set  of  base  terms  (each  assigned  a  type),  terms  are  built 
from  variables  and  base-terms  using  A-abstr action,  application,  as  well  as  by  (i) 
Pairing:  if  Mi,  M2  are  terms  of  types  ti  and  r2  respectively,  then  (Mi,  M2)  is  a 
term  of  type  ri  xr2;  and  (ii)  Projection:  if  P  is  a  term  of  type  ti  xr2  then  IP 
and  2P  are  terms  of  types  ti  and  r2,  respectively. 

Let  RRec(A)  be  lA’^,  with  base  types  0,1,...  (type  £  is  meant  to  denote 
A^),  and  with  the  following  constants:  (1)  for  each  constructor  c*  of  A  and  each 
^  >  0,  a  constant  cf  of  type  (Ti[£]\  (2)  for  each  type  r,  a  constant  R^  =  Rr  of 
type  £, 0-1  [r], . . . <rk[r]-^r^  where  £  =  l-|-max;{m  |  m  occurs  in  r  };  (3)  for  each 
type  r,  a  constant  of  type  o,  /3i[r], . . .  — ^r,  where  pjr]  =  0’’*  — ►r. 

The  reduction  rules  of  RRec(A)  are:  (i)  the  ^-rule;  (ii)  Pair- reductions: 
1{Mi,M2)  Ml,  2 (Ml, M2)  =>  M2;  (iii)  for  each  type  r  the  rule  of  recur¬ 
rence  in  type  r;  and  (iv)  for  each  type  r  the  rule  of  selection  in  type  r: 

St(c^®i  • .  •  ^  M^cii  . . .  cin 

We  write  RReCo(A)  for  the  fragment  of  RRec(A)  with  R,-  only  for  types  r 
without  — 

We  now  reformulate  the  mapping  k  from  IT(A,  P)  to  Rec(A)  as  a  mapping 
from  RT(A,  P)  to  RRec(A).  The  modifications  are  as  follows: 

2^  If  n  derives  (p  by  A-introduction,  then  kII  =  (/till,  /cil2)- 
3^  If  n  derives  (p  by  left- A-eliminat ion,  then  «i7  =  l(/ciTi);  if  77  derives  (p  by 
right- A-elimination,  then  kII  =  2(k7Ti). 

5’  If  77  derives  . .  .trj)  by  the  generative  Ci  rule,  then 

k77  =  cf  (k77i)  •  ■  •  (K:77rJ. 

6’  If  77  derives  <p[t]  by  A-induction  from  A^(t)  and  CJci[An.(p]  {i  =  1...A:), 
then  kII  =  RK(^(/c77i)(/t772)  •  •  *  (kP^+i). 

7  If  77  derives  <p[t]  by  A-selection from  A^(t)  and  CIq. [Xu.(p]  {i  =  1 . .  .k)^  then 
ac77  =  Sk^(«77i)(k772)  •  ■  •  (K77fc+i). 

A  ramified  analog  of  Theorem  2.6  is  proved  now  exactly  as  for  the  un-ramified 
case: 


Theorem  5.1  Suppose  that  the  equational  program  (P,f)  computes  a  function 
f  over  A.  If  II  is  a  deduction  of  RT(A,P)  deriving  Aj(x)  A£(f(x)),  then  kII 
represents  f  in  RRec(A).  Similarly  if  77  is  a  deduction  of  RTo(A),  then  kII 
represents  f  in  RReCo(A). 
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5.2  Characterization  theorems 

Theorem  5.2  A  function  over  a  free  algebra  A  is  provable  in  RT(A)  iff  it  is 
computable  in  Kalmar-elementary  time. 

Proof.  A  function  over  A  provable  in  RT(A)  is  represented  in  RRec(A),  by 
Theorem  5.1,  and  is  therefore  computable  in  elementary  time  by  [Lei94a]. 

Conversely,  if  a  function  is  computable  in  elementary  time,  then  it  is  provable 
in  RT(A)  by  Proposition  4.6.  □ 

Theorem  5.3  A  function  over  W  is  provable  in  RTo(W)  iff  it  is  computable  in 
polynomial  time. 

Proof.  If  a  function  /  over  W  is  provable  in  RTo(W)  then  it  is  represented  in 
RReCo(W),  by  Theorem  5.1,  and  is  therefore  computable  in  polynomial  time  by 
[Lei94c]. 

Conversely,  if  a  function  is  computable  in  polynomial  time,  then  it  is  provable 
in  RT(W)  by  Proposition  4.3.  □ 

Finally,  we  obtain  in  the  same  way 

Theorem  5.4  A  function  over  N  is  provable  in  RTo(N)  iff  it  is  computable  on 
a  multi-tape  Turing-machine  in  linear  space. 


6  Discussion 

We  conclude  with  comments  of  potential  interest  to  the  philosophically  inclined 
reader. 

6.1  Ecto-algebraic  theories 

Peano’s  original  axiomatization  of  arithmetic  [Pea89]  refers  explicitly  to  a  unary 
predicate  iV,  intended  to  denote  the  property  “is  a  natural  number.”  His  axioms 
fall  into  four  groups:  Generative  axioms  for  N,  which  guarantee  that  the  extension 
of  N  includes  the  denotations  of  all  numerals;  Separation  axioms:  Va:  -i(0  =  s(x)) 
and  Va:, 2/  {s{x)^s{y)  — ►  x^y),  which  guarantee  that  the  denotations  of  the 
numerals  are  distinct;  the  defining  equations  for  addition  and  multiplication;  and 
the  schema  of  Induction.  The  generative  axioms  are  redundant  in  the  traditional 
model  theoretic  approach  to  axiomatization,  where  the  universe  of  discourse  is 
assumed  to  be  the  structure  in  hand.  This  approach  might  be  dubbed  endo- 
algebraic,  in  contrast  to  Peano’s  approach,  which  is  ecto-algebraic  in  the  sense 
that  the  intended  structure  is  delineated  within  broader  universes. 

Although  ecto-algebraic  theories  have  been  viewed  for  almost  a  century  as 
un-necessarily  verbose,  they  remain  of  independent  interest:  they  streamline 
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reasoning  about  partial  functions,  support  a  natural  homomorphism  to  typed 
functional  programs,  and  permit  a  transparent  representation  of  the  predicative 
critique  of  Peano  Arithmetic  (see  §6.4  below). 

We  have  generalized  Peano’s  approach  to  arbitrary  free  algebras,  while  drop¬ 
ping  the  separation  axioms  and  the  equational  function  definitions.  Indeed,  the 
separation  axioms  have  no  effect  on  convergence  of  computation  (provided  the 
computation  model  in  hand  does  not  use  branching  on  equality,  which  func¬ 
tional  programs  indeed  do  not).  And  by  refraining  from  function  definitions  we 
bring  out  the  intrinsic  nature  of  the  algebra  in  hand,  rather  than  its  computa¬ 
tional  character  when  suitably  extended;  hence  our  choice  of  the  phrase  intrinsic. 
One  could  wonder,  for  instance,  whether  the  set  Fnc(PA)  of  provably  recursive 
functions  of  PA  is  fundamentally  related  to  the  set  N,  since  if  only  addition 
is  used  then  the  class  of  provably  recursive  functions  is  much  reduced  [Pre29]. 
Godel’s  Theorem  (2.8  above)  puts  doubts  to  rest,  by  linking  Fnc(PA)  solely  to 
recurrence  in  finite  type.  Our  Theorem  2.7  is  the  proof  theoretic  analog  of  that 
result. 


6.2  Existence  and  divergence 

A  number  of  logics,  dubbed  free  logics,  have  been  proposed  for  incorporating 
partial  functions  and  non-denoting  terms  into  formal  reasoning.  Most  of  these 
use  an  existence  predicate  E,  intended  to  hold  true  only  of  the  denoting  terms, 
with  the  rule  of  universal  instantiation  restricted  accordingly:  from  Wx(p[x]  and 
E{t)  infer  There  is  a  choice  as  to  the  free  variables:  they  can  be  interpreted 
as  ranging  over  potentially  non-existing  entities,  or  over  existing  entities  only.^° 
Common  to  all  these  logics  is  the  underlying  semantics  of  a  fixed  universe,  with 
the  predicate  E  used  to  relate  the  scopes  of  free  variables,  of  bound  variables, 
and  of  terms. 

Intrinsic  theories,  on  the  other  hand,  are  based  on  standard  logic,  and  have 
each  a  family  of  standard  models,  in  which  all  terms  are  denoting.  This  is  different 
from  free  logic  in  several  respects.  Convergence  and  denotation  are  treated  here 
as  mathematical  issues,  not  logical  ones,  and  they  do  not  require  any  departure 
from  traditional  deductive  systems.  Computational  divergence  is  reflected  simply 
in  the  canonical  model  not  being  a  model  of  the  program.^^  It  seems  therefore 
plausible  that  intrinsic  axiomatizations  will  prove  to  be  more  useful  than  free 
logics  for  reasoning  about  functional  programs. 


The  first  approach  is  called  outer  or  Meinongian  in  [Lam91],  E-logic  in  [TvD88, 
1.2.2],  and  logic  of  existence  in  [Fef95];  it  is  developed  in  [Sco67,  Sco79].  The  latter 
is  called  inner  or  Russelian  in  [Lam91],  logic  of  partiaJ  terms  in  [Bee85],  -logic  in 
[TvD88, 1.2.2],  and  logic  of  deknedness  in  [Fef95]. 

E-logic  might  be  interpreted  as  an  intrinsic  theory,  with  all  variables  relativized  to 
E,  but  that  is  of  course  a  limitation  on  the  expressiveness  of  the  intrinsic  theory. 
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6.3  Existential  quantification 

The  definition  of  “provably  recursive  functions”  uses  the  existential  quantifier 
in  an  essential  way,  while  the  ecto-algebraic  definition  of  “provable  functions” 
does  not.  The  difference  is  technically  minor,  but  it  continues  a  long  history 
of  doing  away  with  3:  from  formalization  styles  (Hilbert’s  epsilon),  to  model 
theoretic  results  (Skolemization  and  Herbrand’s  Theorem),  to  the  fact  (observed 
in  [CS95])  that  work  in  applied  mathematics  and  theoretical  physics  is  mostly 
equational  and  quantifier-free. 


6.4  Ramification 

The  ramification  of  intrinsic  theories  can  be  motivated  on  grounds  analogous  to 
the  ramification  of  sets  in  predicative  second  order  logic.  The  set  N  of  natural 
numbers  is  implicitly  defined  by  Peano’s  axioms:  the  generative  axioms  convey 
a  lower  bound  on  the  extension  of  N,  and  the  induction  schema  approximates  an 
upper  bound.  However,  as  observed  in  [Nel86],  if  a  formula  has  quantifiers,  then 
its  meaning  presupposes  the  delineation  of  N  as  the  domain  of  the  quantifiers, 
and  therefore  using  induction  over  as  a  component  of  the  delineation  of  N  is  a 
circular  enterprise.  Our  ecto-algebraic  setting  makes  it  possible  to  articulate  this 
critique  precisely  and  to  address  it  directly:  using  induction  over  formulas  that 
refer  to  the  predicate  N  in  order  to  delineate  N  itself  is  circular.  To  circumvent 
this  circularity  one  stratifies  N  into  a  sequence  No,  Ni . . .,  just  as  in  Predicative 
Analysis  one  avoids  impredicative  set  quantification  by  a  stratified  progression 
of  universes  of  sets.^^  Namely,  No  is  intended  as  a  first  approximation  of  N,  with 
induction  over  No  permissible  for  formulas  without  explicit  reference  to  N.  Then, 
Ni+i  is  an  approximation  improving  on  Nf,  since  induction  over  Ni+i  is  allowed 
for  additional  formulas.  Note,  however,  that  the  predicative  critique  does  not 
apply  to  forms  of  induction  that  do  not  depend  on  N  as  a  totality,  notably  the 
Selection  schema. 

A  similar  argument  applies  to  any  free  algebra  A,  leading  to  a  ramificiation 
into  Ao,  Ai . .  The  crucial  bifurcation  is  between  the  first  two  levels,  Aq  and 
Ai .  Taking  A  =  W  for  instance,  an  element  w  eWo  can  be  used  as  a  bit-store: 
one  can  access  the  i’th  bit  oiw,  namely  case  (dstr^^Uw),  €,0^1)]  but  it;  cannot  be 
used  as  a  completed  totality,  i.e.  as  template  for  induct  ion/recursion.  The  use 
of  further  levels  A2,  A3, . . .  contributes  neither  to  the  proof  theoretic  power  of 
RT(A),  nor  to  the  definitional  power  of  RRec(A).  However,  it  does  contribute 
to  succinctness  (see  [Lei94c]  for  ramified  recurrence) 

The  initial  universe  Uo  consists  of  the  first-order  definable  sets,  Ua+i  of  the  sets 
definable  using  set  quantification  over  Up  with  /3  <  a,  and  =df  Ua<e 
ordinals 

The  analogous  ramification  of  recurrence  is  discussed  and  used  in  [Lei94c]. 

Several  authors  have  independently  discovered  the  usefulness  of  data  bifurcation 
[BC92,  Lei90b,  Sim88]. 
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6.5  Feasibility 

The  ramification  of  intrinsic  theories  goes  only  some  way  in  restricting  their 
computational  contents,  as  shown  in  Theorem  5.2.  Indeed,  while  ramification 
excludes  the  circular  definition  of  A,  RT(A)  still  reflects  the  admission  of  each 
level  Ai  as  a  completed  totality:  applying  induction  to  a  formula  of  RT(A) 
presupposes  that  the  meaning  of  (p  has  been  understood,  but  if  Aj  is  referred 
to  negatively  in  y?,  as,  for  example,  in  Vv  Aro(v)  No{e{v,u))  (the  formula 
used  above  in  proving  exponentiation),  then  the  meaning  of  ip  hinges  on  our 
understanding  not  only  of  what  is  in  Aq,  but  also  of  what  is  not. 

It  follows  that  a  “Rnitistic”  view  of  mathematical  existence,  which  requires 
that  infinite  totalities  be  used  only  as  unbounded  processes  but  never  as  com¬ 
pleted  totalities,  corresponds  to  RTo(A).  Thus,  theorems  4.2  and  4.3  indicate  a 
close  kinship  between  feasible  computing  and  a  finitistic  mathematical  ontology. 
A  similar  kinship,  between  poly-time  and  an  articulation  of  the  finitistic  position 
within  second  order  logic,  is  discussed  in  [Lei94b] 
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Abstract.  We  firstly  survey  several  forms  of  Herbrand’s  theorem.  What 
is  commonly  called  “Herbrand’s  theorem”  in  many  textbooks  is  actu¬ 
ally  a  very  simple  form  of  Herbrand’s  theorem  which  applies  only  to 
V3-formulas;  but  the  original  statement  of  Herbrand’s  theorem  applied 
to  arbitrary  first-order  formulas.  We  give  a  direct  proof,  based  on  cut- 
elimination,  of  what  is  essentially  Herbrand’s  original  theorem.  The  “no¬ 
counterexample  theorems”  recently  used  in  bounded  and  Peano  arith¬ 
metic  are  immediate  corollaries  of  this  form  of  Herbrand’s  theorem. 
Secondly,  we  discuss  the  results  proved  in  Herbrand’s  1930  dissertation. 


1  Introduction 

This  paper  discusses  the  famous  theorem  of  Herbrand,  which  is  one  of  the  central 
theorems  of  proof-theory.  The  theorem  called  “Herbrand's  theorem”  in  modern- 
day  logic  courses  is  typically  only  a  very  weak  version  of  the  theorem  originally 
stated  by  Herbrand  in  his  1930  dissertation  [8].  His  1930  dissertation  contains 
in  addition  a  number  of  other  fundamental  results,  including,  the  unification 
algorithm,  the  fact  that  equality  axioms  do  not  help  in  proving  equality-free 
sentences,  a  main  result  that  is  very  similar  to  the  cut-elimination  theorem,  and 
even  a  partial  statement  of  the  completeness  theorem.  The  dissertation  also  con¬ 
tains  a  serious  flaw  in  the  proof  of  the  main  theorem,  which  was  discovered  and 
corrected  by  Dreben  et  al.  in  the  1960’s,  as  well  as  earlier  by  Godel  in  unpub¬ 
lished  work. 

This  author  first  studied  Herbrand’s  thesis  while  preparing  an  introductory 
article  [1];  there  we  restate  Herbrand’s  theorem  in  an  essentially  equivalent  form 
and  give  a  direct  proof  based  on  the  cut-elimination  theorem  (this  restatement 
is  the  same  as  Theorem  3  of  the  present  paper).  Since  Herbrand’s  work  contains 
a  number  of  interesting  constructions  that  are  not  widely  known,  we  felt  it 
worthwhile  to  prepare  this  paper  as  a  survey  of  Herbrand’s  main  results  in 
chapter  5  of  his  dissertation. 

The  outline  of  this  paper  is  follows:  first  we  discuss  the  coimnonly  used, 
weak  form  of  Herbrand’s  theorem  that  applies  only  to  V3-formulas.  Then  we 
discuss  two  ways  of  extending  the  theorem  to  general  formulets:  firstly,  using  Her- 
brand/Skolem  functions  to  reexpress  an  arbitrary  formula  as  a  V3  formula,  and, 
secondly,  using  a  method  based  on  “strong  V-expansions”  to  prove  a  theorem 
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which  is  very  similar  to  the  fundamental  theorem  of  Herbrand.  We  give  proofs 
of  these  results  based  on  the  cut-elimination  theorem  for  the  sequent  calculus. 
After  that,  we  discuss  the  fundamental  theorem  as  stated  by  Herbrand,  We  also 
discuss  the  relationship  of  Herbrand’s  work  to  the  completeness  theorem  and 
the  cut-elimination  theorem.  Finally,  we  briefly  discuss  the  error  in  Herbrand’s 
proof;  for  a  full  discussion  of  this  error  and  its  correction,  the  reader  should 
refer  to  the  papers  by  Dreben  et  al  and  to  Goldfarb  [7]  for  an  account  of  Godel’s 
unpublished  work. 

References  on  Herbrand’s  dissertation  include  the  dissertation  itself  [8],  the 
translation  of  its  fifth  chapter  and  the  accompanying  notes  by  Dreben  and  van 
Heijenoort  [9].  Discussions  of  the  errors  in  Herbrand’s  thesis  can  be  found  in  the 
papers  by  Dreben  et  al.  [3,4,5]  and  in  Goldfarb  [7].  Herbrand’s  collected  works 
are  contained  in  [10,11].  Goldfarb  [6]  has  further  discussion  of  the  history  of 
Herbrand’s  theorem  and  an  application  to  incompleteness. 

All  proofs  in  this  paper  are  presented  in  terms  of  the  sequent  calculus;  how¬ 
ever  for  space  reasons,  background  material  and  definitions  for  the  sequent  calcu¬ 
lus  are  not  included  in  this  paper.  A  reader  unfamiliar  with  the  sequent  calculus 
should  either  skip  all  proofs  or  refer  to  [1,17]  for  definitions. 

We  are  grateful  to  R.  Parikh  and  W.  Goldfarb  for  comments  on  an  earlier 
draft  of  this  paper. 


2  The  weak  form  of  Herbrand’s  theorem 


Herbrand’s  theorem  is  one  of  the  fundamental  theorems  of  mathematical  logic 
and  allows  a  certain  type  of  reduction  of  first-order  logic  to  propositional  logic. 
In  its  simplest  form  it  states: 


Theorem  1.  Let  T  be  a  theory  axiomatized  by  purely  universal  formulas.  Sup¬ 
pose  that  T  1=  (Vx)(32/i,  . .  .,2/jfc)5(x,y)  with  B(x,y)  a  quantifier-free  formula. 
There  there  is  a  finite  sequence  of  terms  Uj  =  with  I  <  i  <  r  and 

1  <  j  <  k  so  that 


T  I-  (Vx) 


j  •  •  •  yl'ijk) 


It  is  well-known  how  to  give  a  model-theoretic  proof  of  Theorem  1;  it  is  also 
straightforward  to  give  a  constructive,  proof-theoretic  proof  based  on  the  cut- 
elimination  theorem  as  follows: 


Proof  Since  T  is  axiomatized  by  purely  universal  formulas,  it  may,  without  loss 
of  generality,  be  axiomatized  by  quantifier-free  formulas  (obtained  by  removing 
the  universal  quantifiers).  Let  %  denote  the  set  of  sequents  of  the  form  — with 
A  a  (quantifier-free)  axiom  of  T .  Define  a  LK<i  proof  to  be  a  sequent  calculus 
proof  in  Gentzen’s  system  LK,  except  allowing  sequents  fromT  in  addition  to 
the  usual  initial  sequents.^  Since  T  (Vx)(3y)R(x,  y),  there  is  a  LATx-proof  of 
the  sequent  -^(3y)H(a, y). 

^  T'A’x  may  optionally  contain  equality  axioms  as  initial  sequents. 


197 


By  the  free-cut  elimination  theorem,  there  is  a  free-cut  free  LK^-proof  P  of 
this  sequent,  and  since  the  ‘T-sequents  contain  only  quantifier-free  formulas,  all 
cut  formulas  in  P  are  quantifier-free.  Thus,  any  non-quantifier-free  formula  in  P 
must  be  of  the  form  (Bi/j)  •  ♦  •(3j/fc)B(a,ti, . . . j/j , . .  .,2/*;)  with  I  <  j  <  k. 
We  claim  that  P  can  be  modified  to  be  a  valid  proof  of  a  sequent  of  the  form 

The  general  idea  is  to  remove  all  3:right  inferences  in  P  and  remove  all  existential 
quantifiers,  replacing  the  bound  variables  by  appropriate  terms.  Since  there  may 
have  been  contractions  on  existential  formulas  that  are  no  longer  identical  after 
terms  are  substituted  for  variables  it  will  also  be  necessary  to  remove  contractions 
and  add  additional  formulas  to  the  sequents.  To  do  this  more  formally,  we  know 
that  any  sequent  in  P  is  of  the  form  P — A'  (up  to  order  of  the  formulas  in 
the  sequent),  where  each  formula  in  P  and  A  is  quantifier-free  and  where  each 
formula  in  A*  is  not  quantifier-free  but  is  purely  existential.  We  can  then  prove 
by  induction  on  the  number  of  lines  in  the  free-cut  free  proof  of  P^A^A'  that 
there  is  an  r  >  0  and  a  cedent  zi"  of  the  form 

•  •  • ) 

such  that  r— A^‘  is  provable.  We  leave  the  rest  of  the  details  to  the  reader. 

0 

We  define  an  instance  of  a  universal  formula  (Vx)A(x)  to  be  any  quantifier- 
free  formula  A(t).  It  is  not  hard  to  see  using  cut  elimination,  that  if  a  quantifier- 
free  formula  C  is  a  consequence  of  a  universal  theory  T,  then  it  is  a  tautological 
consequence  of  some  finite  set  of  instances  of  axioms  of  T  and  of  equality  ax¬ 
ioms.  In  the  special  case  where  T  is  the  null  theory,  we  have  that  C  is  a  conse¬ 
quence  of  instances  of  equality  axioms  (and  C  is  therefore  called  a  quasHautol- 
ogy).  If,  in  addition,  C  does  not  involve  equality,  C  will  be  tautologically  valid. 
Thus,  Herbrand’s  theorem  reduces  provability  in  first-order  logic  to  generation 
of  (quasi)tautologies. 

The  weak  form  of  Herbrand’s  theorem  stated  above  as  Theorem  1  has  lim¬ 
ited  applicability  since  it  applies  only  to  V3-consequences  of  universal  theories: 
fortunately,  however,  there  are  several  ways  to  extend  Herbrand’s  theorem  to 
more  general  situations.  In  section  3  below,  we  explain  one  such  generalization; 
but  first  we  give  a  simpler  method  of  widening  the  applicability  of  Herbrand’s 
theorem,  based  on  the  introduction  of  new  function  symbols,  which  we  call  Her- 
brand  and  Skolem  functions,  that  allow  quantifier  alternations  to  be  reduced. 

For  notational  simplicity,  we  will  consider  only  formulas  in  prenex  normal 
form  for  the  rest  of  this  section;  however,  the  definitions  and  theorem  below  can 
be  readily  generalized  to  arbitrary  formulas. 

Definition  2.  Let  (3a;)7l(a;,  c)  be  a  formula  with  c  all  of  its  free  variables.  The 
Skolem  function  for  (3a;)i4  is  represented  by  a  function  symbol  fsxA  and  has  the 
defining  axiom: 

Sk-deJifa^A)  ■  (Vy)(Va:)  (A(x,y)  A(/3x^(y),y)) . 
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Note  that  Sk-^deJ{f3^A)  implies  (Vy)  ((3ic)A(a;, y)  ^  A(f3xA{y),y))^ 

Definitions.  Let  >i(c)  be  a  formula  in  prenex  form.  The  Skolemizaiion,  vl‘^(c), 
of  A  is  the  formula  defined  inductively  by: 

(1)  If  A{c)  is  quantifier-free,  then  i4‘^(c)  is  -A(c). 

(2)  If  yl(c)  is  (V2/)5(c,2/),  then  j4’^(c)  is  the  formula  (V2/)5‘^(c,  j/). 

(3)  If  A(c)  is  (3?/)B(c,2/),  then  A'^(c)  is  5*^(0, /a (c)),  where  /a  is  the  Skolem 
function  for  A. 

It  is  a  simple,  but  important  fact  that  A^  1=  A. 

The  Skolemizaiion  of  a  theory  T  is  the  theory  =  {^4*^  :  A  Q  T}.  Note 
that  is  a  purely  universal  theory.  Incidentally,  the  set  of  axioms  of  the 

Skolem  functions  can  be  equivalently  expressed  as  a  set  of  universal  formulas; 
however,  they  are  not  included  in  theory  .  From  model-theoretic  considera¬ 
tions,  it  is  not  difficult  to  see  that  contains  and  is  conservative  over  T. 

We  next  define  the  concept  of  ‘Herbrandization’  which  is  completely  dual  to 
the  notion  of  Skolemization: 

Definition  4.  Let  (Vx)yl(aj,  c)  be  a  formula  with  c  all  of  its  free  variables.  The 
Herbrand  function  for  (Var).A  is  represented  by  a  function  symbol  h\/xA  and  has 
the  defining  axiom: 


(Vy)(Va:)  {-^A{x,y)  -.^(/iv.A(y),y))  • 

Note  that  this  implies  (Vy)  ((Va;)yl(a:,y)  A{h^xA{y),y))-  The  Herbrand  func¬ 

tion  can  also  be  thought  of  as  a  ‘counterexample  function’;  in  that  (Vic)A(x)  is 
false  if  and  only  if  h\/xA  provides  a  value  x  which  is  a  counterexample  to  the 
truth  of 

Definitions.  Let  A(c)  be  a  formula  in  prenex  form.  The  Herbrandization, 
A^{c),  of  A  is  the  formula  defined  inductively  by: 

(1)  If  7l(c)  is  quantifier-free,  then  ^^(c)  is  A(c). 

(2)  If  ^(c)  is  {3y)B{c,y),  then  A^(c)  is  the  formula  {3y)B^ (c,y). 

(3)  If  .A(c)  is  (V?/)H(c,  2/),  then  A^{c)  is  H-^(c,  /ia(c)),  where  is  the  Herbrand 
function  for  A. 

It  is  not  hard  to  see  that  A  ^  A^ .  Note  that  A^  is  purely  existential. 


Theorem  6.  Lei  T  be  set  of  prenex  formulas  and  A  any  prenex  formula.  Then 
the  following  are  equivalent: 

(1) TI=A 

(2)  T^\=  A, 

(3)  T  1=  A^, 

(4)  1=  A^, 


199 


This  theorem  is  easily  proved  from  the  above  definitions  and  remarks.  The 
importance  of  Theorem  6  lies  in  the  fact  that  is  a  universal  theory  and  that 
is  an  existential  formula,  and  that  therefore  Herbrand’s  theorem  applies  to 
TS  fz  Thus,  Theorem  6  allows  Theorem  1  to  be  applied  to  an  arbitrary 
logical  implication  T  t=  A,  at  the  cost  of  converting  formulas  to  prenex  form  and 
introducing  Herbrand  and  Skolem  functions. 

3  A  strong  form  of  Herbrand’s  theorem 

Herbrand  actually  proved  a  much  more  general  theorem  than  Theorem  1  which 
applies  directly  whenever  1=  A,  for  ^  a  general  formula,  not  necessarily  V3.  His 
result  also  avoids  the  use  of  Skolem/Her brand  functions.  The  theorem  we  state 
next  is  quite  similar  in  spirit  and  power  to  the  theorem  as  stated  originally  by  [8]. 

In  this  section,  we  shall  consider  a  first-order  formula  A  such  that  A. 
Without  loss  of  generality,  we  shall  suppose  that  the  propositional  connectives 
in  A  are  restricted  to  be  A,  V  and  -i,  and  that  the  connective  appears  only 
in  front  of  atomic  subformulas  of  A.  (The  only  reason  for  this  convention  is 
that  it  avoids  having  to  keep  track  of  whether  quantifiers  appear  positively  and 
negatively  in  A.) 

Definition?.  Let  A  satisfy  the  above  convention  on  negations.  An  y -expansion 
of  A  is  any  formula  that  can  be  obtained  from  A  by  a  finite  number  of  applica¬ 
tions  of  the  following  operation: 

(a)  If  5  is  a  subformula  of  an  V-expansion  A'  of  A,  replacing  B  in  A'  with  By  B 
produces  another  V-expansion  of  A. 

A  strong  y -expansion  of  A  is  defined  similarly,  except  that  now  the  formula  B  is 
restricted  to  be  a  subformula  with  outermost  connective  an  existential  quantifier. 

Definitions.  Let  A  be  a  formula.  A  prenexification  of  A  is  a  formula  obtained 
from  A  by  first  renaming  bound  variables  in  A  so  that  no  variable  is  quantified 
more  than  once  in  A  and  then  using  prenex  operations  to  put  the  formula  in 
prenex  normal  form. 

Note  that  there  will  generally  be  more  than  one  prenexification  of  A  since  prenex 
operations  may  be  applied  in  different  orders  resulting  in  a  different  order  of  the 
quantifiers  in  the  prenex  normal  form  formula. 

Definition  9.  Let  A  be  a  valid  first-order  formula  in  prenex  normal  form,  with 
no  variable  quantified  twice  inA.  IfAhasr>0  existential  quantifiers,  then  A  is 
of  the  following  form  with  B  quantifier-free: 

(Vxi  •  •  •®„i)(3j/i)(Vx„i+i  •  •  ■Xn^){3y2)  •  •  •(3s/r)(Vs„,+i  •  •  •x„,^,)B(x,y) 

with  0  <  ni  <  n2  <  •  •  ♦  <  Wr+i-  A  witnessing  substitution  for  A  is  a  sequence 
of  terms  (actually,  semiterms)  <i, . .  ,tr  such  that  (1)  each  U  contains  arbitrary 
free  variables  but  only  bound  variables  from  . . .  ,Xni  and  (2)  the  formula 
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5(x,fi,  is  a.  quasitautology  (i.e.,  a  tautological  consequence  of  instances 

of  equality  axioms  only).  In  the  case  where  B  does  not  contain  the  equality  sign, 
then  (2)  is  equivalent  to  B  being  a  tautology. 

Let  T  be  a  first-order  theory.  A  sequence  of  terms  is  said  to  witness  A  over  T 
if  the  above  conditions  hold  except  with  condition  (2)  replaced  by  the  weaker 
condition  that  T  \=  (Vx)B(a;,  t). 

Definition  10.  A  Herbrand  proof  oi  a  first-order  formula  A  consists  of  a  prenex- 
ification  A*  of  a  strong  V-expansion  of  A  plus  a  witnessing  substitution  a  for  A*. 

A  Herbrand  T-proof  of  A  consists  of  a  prenexification  A*  of  a  strong  V- 
expansion  of  A  plus  a  substitution  which  witnesses  A  over  T. 

We  are  now  in  a  position  to  state  the  general  form  of  Herbrand ’s  theorem: 

Theorem  11,  A  first-order  formula  A  is  valid  if  and  only  if  A  has  a  Herbrand 
proof  More  generally,  ifT  is  a  universal  theory,  then  T  1=  A  if  and  only  if  A  has 
a  Herbrand  T-proof 

Proof  We  shall  sketch  a  proof  of  only  the  first  part  of  the  theorem  since  the 
proof  of  the  second  part  is  almost  identical.  Of  course  it  is  immediate  from  the 
definitions  that  if  A  has  a  Herbrand  proof,  then  A  is  valid.  So  suppose  A  is  valid, 
and  therefore  has  a  cut-free  LA'-proof  P.  We  shall  modify  P  in  stages  so  as  to 
extract  a  Herbrand  proof  of  P. 

The  first  stage  will  involve  restricting  the  formulas  which  can  be  combined 
by  a  contraction  inference.  In  order  to  properly  keep  track  of  contractions  of 
formulas  in  a  sequent  calculus  proof,  we  must  be  careful  to  formulate  inference 
rules  with  two  hypotheses  in  a  “multiphcative”  fashion  so  as  to  avoid  the  problem 
of  having  implicit  contractions  on  side  formulas  in  inferences  with  two  hypothesis 
such  as  V:left  and  A:right.  For  example,  we  want  to  formulate  the  \/:left  inference 
rule  in  the  multiplicative  form 

A,r^A  B,r'^A' 

AV  B,r,r'-^A,A' 

rather  than  in  the  “additive”  form 

A,r-^A  B,r~^A 

AyB,r~-^A 

since  the  additive  form  contains  implicit  contractions  on  side  formulas  in  P  and  A, 
whereas  the  multiplicative  formulation  does  not  contain  implicit  contractions. 
We  also  use  analogous  multiplicative  formulations  of  the  A:right  and  cut  rule. 
Of  course,  using  multiplicative  formulations  rules  instead  of  additive  formulation 
does  not  change  the  strength  of  the  sequent  calculus,  since  either  form  may  be 
derived  from  the  other  with  the  use  of  weak  structural  inferences.  Furthermore, 
the  cut-elimination  and  free-cut  elimination  theorems  hold  with  either  formula¬ 
tion.  We  therefore  henceforth  use  the  multiplicative  formulation  of  the  rules  of 
inference  for  the  sequent  calculus. 
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A  contraction  inference  is  said  to  be  a  propositional  contraction  (resp.,  an  3- 
contraction  provided  that  the  principal  formula  of  the  contraction  is  quantifier- 
free  (resp.j  its  outermost  connective  is  an  existential  quantifier).  The  first  step 
in  modifying  P  is  to  form  a  cut-free  proof  Pi ,  also  with  endsequent  -^A  such 
that  all  contraction  inferences  in  Pi  are  propositional  or  3-contractions.  The 
construction  of  Pi  from  P  is  done  by  a  “contraction-elimination”  procedure.  For 
this  purpose,  we  define  the  E-depth  of  a  formula  by  letting  the  P-depth  of  a 
quantifier-free  formula  or  a  formula  which  begins  with  an  existential  quantifier  be 
equal  to  zero,  and  defining  the  P- depth  of  other  formulas  inductively  by  letting 
the  P-depth  of  equal  the  P-depth  of  <p  plus  one  and  by  letting  the  P-depths 
of  (p\/  and  A  ^  equal  one  plus  the  maximum  of  the  P-depths  of  <p  and  ip. 
Then  we  prove,  by  double  induction  on  the  maximum  P-depth  d  of  contraction 
formulas  and  the  number  of  contractions  of  formulas  of  this  maximum  P-depth, 
that  Pi  can  be  transformed  into  a  proof  in  which  all  contractions  are  on  formulas 
of  P-depth  zero.  The  induction  step  consists  of  removing  a  topmost  contraction 
inference  of  the  maximum  P-depth  d.  For  example,  suppose  that  the  following 
inference  is  a  topmost  contraction  with  principal  formula  of  P-depth  d\ 

r->Z\,(Vx)P,(Vx)P 

P-^Z\,(Vx)P 

Since  Pi  is  w.l.o.g.  in  free  variable  normal  form  and  since  this  is  a  topmost 
contraction  of  P-depth  d,  we  can  modify  the  subproof  R  of  Pi  by  removing  at 
most  two  inferences  and/or  changing  some  Weakening:right  infeiences  to 

get  a  proof  of  P—yA,  B{a),  P(a'),  where  a  and  a'  are  free  variables  not  appearing 
in  the  endsequent  of  R.  Further  replacing  a'  everywhere  by  a  gives  a  proof  of 
P—^A,  B{a),  B{a):  we  use  this  get  to  a  proof  ending: 


P^A,B{a),B{a) 

P-^A,B{a) 

P~^A,  {yx)B 

Thus  we  have  reduced  the  P-depth  of  the  contraction  inference.  A  similar  proce¬ 
dure  works  for  contractions  of  P-depth  d  with  outermost  connective  a  proposi¬ 
tional  connective — we  leave  the  details  to  the  reader.  Note  that  the  construction 
of  Pi  depends  on  the  fact  that  propositional  inferences  and  'i .'right  inferences  can 
be  pushed  downward  in  the  proof.  It  is  not  generally  possible  to  push  3:righi 
inferences  downward  in  a  proof  without  violating  eigen  variable  conditions. 

The  second  step  in  modifying  P  is  to  convert  Pi  into  a  cut-free  proof  P2  of 
some  strong  V-expansion  A'  of  A  such  that  every  contraction  in  P2  is  proposi¬ 
tional.  This  is  done  by  the  simple  expedient  of  replacing  every  3-contraction  in 
Pi  with  an  W '.right  inference,  and  then  making  the  induced  changes  to  all  de- 
scendents  of  the  principal  formula  of  the  inference.  More  precisely,  starting  with 
a  lowermost  3-contraction  in  Pi ,  say 
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r-^A,{3x)B,{3x)B 

r~^A,{3x)B 

replace  this  with  an  y.’left  inference 

r  — ^A,  (3a?)jB,  (3aj)J3 
~7-^A,{3x)B\/{3x)B 

and  then,  in  order  to  get  a  syntactically  correct  proof,  replace,  as  necessary, 
subformulas  {3x)B'  of  formulas  in  P  with  {3x)B'  V  {3x)B'  (we  use  the  notation 
B'  since  terms  in  B  may  be  different  in  its  descendents).  Iterating  this  process 
yields  the  desired  proof  P2  of  a  strong  V-expansion  A'  of  A.  By  renaming  bound 
variables  in  P2  we  can  assume  w.l.o.g.  that  no  variable  is  quantified  twice  in  any 
single  sequent  in  P2. 

Thirdly,  from  P2  we  can  construct  a  prenexification  A*  of  A*  together  with 
a  witnessing  substitution,  thereby  obtaining  a  Herbrand  proof  of  A.  To  do  this, 
we  iterate  the  following  procedure  for  pulling  quantifiers  to  the  front  of  the 
proved  formula.  Find  any  lowest  quantifier  inference  in  P2  which  has  not  already 
been  handled:  this  quantifier  inference  corresponds  to  a  unique  quantifier,  (Qa;), 
appearing  in  the  endsequent  of  the  proof  (and  conversely,  each  quantifier  in  the 
endsequent  of  the  proof  corresponds  to  a  unique  quantifier  inference,  since  all 
contraction  formulas  are  quantifier-free).  Use  prenex  operations  to  pull  {Qx)  as 
far  to  the  front  of  the  endsequent  formula  as  possible  (but  not  past  the  quantifiers 
that  have  already  been  moved  to  the  front  of  the  endsequent  formula).  Also, 
push  the  quantifier  inference  downward  in  the  proof  until  it  reaches  the  group  of 
quantifier  inferences  that  have  already  been  pushed  downward  in  the  proof.  It  is 
straightforward  to  check  that  this  procedure  preserves  the  property  of  having  a 
syntactically  valid  proof.  When  we  are  done  iterating  this  procedure,  we  obtain 
a  proof  P3  of  a  prenexification  -^A*  of  A.  It  remains  to  define  a  witnessing 
substitution  for  A*:  this  is  now  easy,  for  each  existential  quantifier  (3y,)  in  A*, 
find  the  corresponding  3:right  inference 

r~^A,B(ti) _ 

r-^A,{3yi)B{yi) 

and  let  the  term  U  be  from  this  inference.  That  this  is  a  witnessing  substitution 
for  A*  is  easily  proved  by  noting  that  by  removing  the  3:right  inference  from  P3, 
a  proof  of  Alf{x,t)  is  obtained  where  A^f  is  the  quantifier-free  portion  of  A*. 

□ 

The  above  theorem  can  be  used  to  obtain  the  following  ‘no- counterexample 
interpretation’  which  has  been  very  useful  recently  in  the  study  of  bounded 
arithmetic  (see  [12,2, 18]). 

Corollary  12.  Let  T  he  a  universal  theory  and  suppose  T  (3a;)(V2/)A(a7,  ?/,  c) 
with  A  a  quantifier-free  formula.  There  is  a  k  >  0  and  terms  ti{c),  t2{c,yi), 

*  This  corollary  is  named  after  the  more  sophisticated  no-counterexample  interpreta¬ 
tions  of  [13,14]. 
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i3(c,2/i,y2),--.,<fc(c,2/i,---yib-i)  such  that 

TI=(Vyi)[^(ti(c),2/i,c) 

V(V2/2)[^(^2(c,2/i),y2,c) 

V(Vy3)[^(^3(c,  yi,  y2),  y3,  c) 

V  •  •  ♦  V  (Vyjb)[A(4(c,  yi, . . . ,  yfc-i),  yfc ,  c))]  •  •  •]]] 

To  prove  the  corollary,  note  that  the  only  strong  V-expansions  of  A  are  for¬ 
mulas  of  the  form  \/{3x){^y)A{x,  y,  c)  and  apply  the  previous  theorem. 


4  No  recursive  bounds  on  number  of  terms 

It  is  interesting  to  ask  whether  it  is  possible  to  bound  the  value  of  r  in  Theorem  1. 
For  this,  consider  the  special  case  where  the  theory  T  is  empty,  so  that  we  have 
an  LfC-proof  P  of  (3xi, . . . ,  Xfc)B(a,x)  where  B  is  quantifier-free.  There  are  two 
ways  in  which  one  might  wish  to  bound  the  number  r  needed  for  Herbrand’s 
theorem:  as  a  function  of  the  size  of  P,  or  alternatively,  as  a  function  of  the  size 
of  the  formula  (3x)P.  For  the  first  approach,  it  follows  immediately  from  the 
proof  of  the  cut-elimination  theorem  in  [1]  and  the  proof  of  Herbrand^s  theorem, 
that  r  <  22[^p||,  where  2f  is  defined  inductively  by  2g  =  a;  and  =  2^?  and 
where  ||Pl|  equals  the  number  of  strong  inferences  in  P.  For  the  second  approach, 
we  shall  sketch  a  proof  below  that  r  can  not  be  recursively  bounded  as  a  function 
of  the  formula  (3x)P.  The  proof  is  based  on  the  unification  algorithm  contained 
in  Herbrand  [9,  para.  2.4] 

To  show  that  r  cannot  be  recursively  bounded  as  a  function  of  (3x)P,  we 
shall  prove  that  having  a  recursive  bound  on  r  would  give  a  decision  procedure 
for  determining  if  a  given  existential  formula  is  valid.  Since  it  is  well  known 
that  validity  of  existential  first-order  formulas  is  un decidable;  this  implies  that 
r  cannot  be  recursively  bounded  in  terms  of  the  formula  size. 

What  we  shall  show  is  that,  given  a  formula  P  as  in  Theorem  1  and  given 
an  r  >  0,  it  is  decidable  whether  there  are  terms  <i,i, . . .  which  make  the 
formula 

r 

(1) 

»=1 

a  tautology.  (This  fact  was  first  proved  by  Herbrand  by  the  same  argument 
that  we  sketch  here.)  This  will  suffice  to  show  that  r  cannot  be  recursively 
bounded.  The  quantifier-free  formula  B  is  expressible  as  a  Boolean  combination 
C(jDi,  . . . ,  Di)  where  each  Dj  is  an  atomic  formula  and  C(*  •  •)  is  a  propositional 
formula.  If  the  formula  (1)  is  a  tautology,  it  is  by  virtue  of  certain  formulas 
Dj(a,tj,i, . . .  being  identical.  That  is  to  say  there  is  a  finite  set  X  of  equal¬ 
ities  of  the  form 
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such  that,  any  set  of  terms  which  makes  all  the  equalities  in  X  true 

will  make  (1)  a  tautology. 

But  now  the  question  of  whether  there  exist  terms  which  satisfy 

such  a  finite  set  X  of  equations  is  easily  seen  to  be  a  first-order  unification 
problem.  The  algorithm  for  solving  first-order  unification  problems  is  given  in 
Herbrand  s  thesis  and  is  now-a-days  well-known;  Robinson  [16]  gives  a  method 
of  getting  a  most  general  solution,  and  Paterson- Wegman  [15]  give  a  linear-time 
algorithm  for  unification.  This  algorithm  either  determines  that  no  choice  of 
terms  will  satisfy  all  the  equations  in  X  or  will  find  a  (most  general)  set  of  terms 
that  satisfy  the  equations  of  X, 

Since,  for  a  fixed  r  >  0,  there  are  only  finitely  many  possible  sets  X  of  equal¬ 
ities,  we  have  the  following  algorithm  for  determining  if  there  are  terms  which 
make  (1)  a  tautology:  for  each  possible  set  X  of  equalities,  check  if  it  has  a  solu¬ 
tion  (i.e.,  a  most  general  unifier),  and  if  so,  check  if  the  equalities  are  sufficient 
to  make  (1)  a  tautology.  □ 

5  The  actual  theorem  of  Herbrand 

In  this  final  section,  we  discuss  the  results  contained  in  chapter  5  of  Herbrand’s 
Ph.D.  thesis.  The  fundamental  theorem  of  this  chapter  is  very  similar  to  Theorem  3 
but  differs  in  some  details.  We  also  describe  the  two  proof  systems,  now  called 
Qh  and  that  Herbrand  used.  The  results  stated  by  Herbrand  include  a  ver¬ 
sion  of  the  cut-elimination  theorem  and  his  proof  methods  give  (or  nearly  give)  a 
version  of  the  completeness  theorem.  There  was  also  a  fairly  serious  error  in  Her¬ 
brand’s  proof,  which  was  first  described  in  published  material  by  Dreben  et  al.; 
this  error  was  apparently  also  recognized  by  Bernays  in  the  1930’s  and  was  dis¬ 
covered  and  corrected  by  Godel  in  unpublished  notes  (see  [7]).  These  errors  in 
no  way  detract  from  the  importance  of  Herbrand’s  work,  since  alternative  proofs 
could  be  given.  In  any  event,  although  there  are  some  false  lemmas  in  Herbrand’s 
work,  his  main  theorems  are  all  fully  correct. 

5.1  Herbrand’s  fundamental  theorem. 

Herbrand’s  fundamental  theorem  applied  to  arbitrary  first-order  formulas  A; 
in  particular,  A  need  not  be  in  prenex  normal  form.  By  renaming  variables, 
one  can  assume  that  no  variable  is  quantified  more  than  once  in  A  and  that  no 
variable  occurs  both  free  and  bound  in  A,  Herbrand  took  prenex  operations  as 
fundamental  in  his  proof  theory  (see  the  definitions  of  Qjj  and  below).  His 
formal  system  allowed  prenex  operations  to  be  applied  not  only  in  a  ‘forward’ 
direction  which  brings  quantifiers  to  the  front  of  a  formula,  but  also  in  a  ‘reverse’ 
direction  pushing  quantifiers  further  inward  in  a  formula.  Herbrand  noted  that 
for  every  formula  ^4  there  is  a  unique  formula,  called  the  canonical  form  of  A, 
which  is  obtained  by  applying  prenex  operations  to  subformulas  of  A  to  push 
quantifiers  as  far  inward  as  possible.  Let  M  be  obtained  from  A  by  erasing  all 
quantifiers  from  A.  Since  we  use  only  connectives  -i,  V  and  A  (Herbrand  used 
only  the  first  two)  and  because  of  our  conventions  on  not  reusing  variables,  it 
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is  easy  to  see  that  any  prenex  formula  obtained  from  A  by  prenex  operations 
consists  of  M  preceded  by  a  string  of  quantifiers.  Thus  all  prenexifications  of  A 
differ  only  in  the  order  of  the  quantifiers. 

We  now  describe  a  tree  expansion  of  A  to  consist  of  a  finite  set  (also  called  a 
forest)  of  labeled  trees:  each  tree  has  its  leaves  labeled  with  the  formula  M  and 
has  its  internal  nodes  labeled  with  quantifiers  (3aj)  or  (Vx)  which  already  appear 
in  A.  Furthermore,  the  following  properties  should  hold: 

1.  For  any  simple  path  from  a  root  of  a  tree  to  a  leaf,  if  the  labels  on  the  path 

are  concatenated,  then  one  obtains  a  formula  which  is  equivalent  to  A  and 
is  obtainable  from  A  by  prenex  operations  only. 

2.  The  trees  are  finite  in  that  each  internal  node  has  only  finitely  many  children. 

3.  If  a  node  has  more  than  one  child,  then  none  of  its  children  are  labeled  with 

universal  quantifiers.® 

To  given  an  example,  consider  a  formula  of  the  form 

(Va:)[(32/)(Vz)A(ar,  y,  z)  V  (3w)5(ar,  u)]. 

One  possible  set  of  trees  associated  to  this  formula  is: 


where  M  is  A(a:,y,  z)  V  B{x,u).  Herbrand  used  a  tabular  notation  to  represent 
this  situation;  namely,  for  this  example,  he  would  write 

Ax  -u  ~y  +z 
Ax  —y  -~u  Az 
-f-a:  ~y  Az  —u 

using  Ax  to  mean  (Va:)  and  —  u  to  mean  (3y),  etc.  Note  that  each  line  in  the  table 
corresponds  to  a  path  in  the  tree.  To  make  the  tree  structure  clearer,  Herbrand 
then  rewrites  the  table  above  as: 
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The  concept  of  a  proposition  derived  from  A  is  defined  as  follows:  for  each  node 
in  the  tree  assign  a  formula  as  follows:  assign  the  matrix  M  to  every  leaf  node, 
and  assign  to  an  internal  node  a  labels  with  {Qv)  the  formula 

(Qt;)[PiV...VPn] 

where  Pi, . . Pm  are  the  formulas  assigned  to  the  n  children  of  a.  Finally,  take 
the  disjunction  of  the  formulas  assigned  to  all  the  roots  of  trees  in  the  forest,  then 
rename  variables  so  that  no  variable  is  used  twice  in  this  disjunction  and  form 
an  arbitrary  prenexification  of  this  disjunction;  the  result  is  called  a  proposition 
derived  from  A. 

It  is  clear  that  a  proposition  derived  from  A  is  equivalent  to  A,  since  it  is 
obtained  by  using  only  the  following  types  of  operations:  (a)  prenex  operations, 
(b)  variable  renamings,  and  (c)  replacing  subformulas  Z  with  Z  \J  Z  (i.e.,  V- 
expansion  steps).  Herbrand’s  fundamental  theorem  can  now  be  stated  as  follows 
(the  theory  Qh  is  described  below;  since  it  is  sound  and  complete,  A  is  Qh- 
provable  iff  A  is  valid): 

Theorem  13.  A  is  provable  in  the  theory  Qjj  iff  there  is  a  proposition  derived 
from  A  which  has  a  witnessing  substitution. 

5.2.  Herbrand’s  proof  systems 

Herbrand’s  thesis  primarily  used  a  proof  system  which  we  shall  denote  Qh] 
he  also  used  a  modified  version,  and  his  fundamental  theorem  states  that 
provability  in  Qh  is  equivalent  to  provability  in  Q'jj.  Formulas  in  these  proof 
systems  involve  the  logical  connectives  V,  V  and  3;  other  symbols,  such  as  —)■ 
are  abbreviations  for  more  complex  formulas.  It  is  not  permitted  for  a  variable  to 
be  quantified  twice  in  a  formula,  or  to  appear  both  free  and  bound  in  a  formula. 
The  system  Qh  has  all  tautologies  as  axioms  and  has  the  following  rules  of 
inference: 

1.  Modus  Ponens;  from  A  and  A  ^  B,  infer  B. 

2.  Rule  of  Simplification:  If  Z'  is  an  alphabetic  variant  of  Z,  then  Z  may  be 

inferred  from  ZV  Z'. 

3.  Universal  Generalization:  from  infer  (Va:)0. 

4.  Existential  Instantiation:  from  ^(i),  infer  (3a:)^(a?). 

5.  The  Rules  of  Passage:  consider  the  following  six  pairs  of  logically  equivalent 

formulas: 


-tBx0  <=>  Va:(-»<?) 
(Va:^)VZ  Vx(^VZ) 
ZV(Vx^)  yx{Z\/0) 
(3aj^)VZ  ^  3a;(^VZ) 
Z\/{3x<h)  ^  3x{ZV^) 
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There  are  twelve  rules  of  passage]  these  allow  a  formula  B  to  be  inferred  from 
the  formula  A  provided  B  is  obtained  from  A  by  replacing  an  occurrence  of  a 
subformula  in  A  which  is  in  one  of  the  above  twelve  forms  with  the  equivalent 
subformula  given  in  the  above  table.  (Note  that  the  conventions  on  variable 
usage  imply  that  x  does  not  appear  in  Z.) 

Herbrand’s  second  proof  system,  which  we  call  is  obtained  from  Qh  by 
disallowing  the  rule  of  modus  ponens,  and  replacing  the  rule  of  simplification 
by  the  generalized  rule  of  simplificaiion  which  permits  B  to  be  inferred  from  A 
when  B  is  obtained  from  A  by  replacing  a  subformula  of  the  form  Z  V  Z'  with 
the  subformula  Z^  provided  Z'  is  an  alphabetic  variant  of  Z. 

A  corollary  of  Herbrand’s  fundamental  theorem  is  the  statement  that  a  for¬ 
mula  is  Q/f-provable  if  and  only  if  it  is  Q'j^-provable.  This  is  a  very  intrigumg 
fact,  since  it  is  evident  that  is  very  similar  to  a  cut-free  sequent  calculus 
proof  system;  in  particular,  there  is  an  analogue  of  the  subformula  property  of 
the  sequent  calculus  which  holds  for  ;  namely,  if  one  measures  the  complexity 
of  formula  in  terms  of  the  depth  of  quantifier  nesting  in  the  canonical  form  of  a 
formula,  then  it  is  evident  that  all  the  formulas  which  appear  in  a  Q|^-proof  of  a 
formula  A  have  complexity  no  greater  than  the  complexity  of  A.  Gentzen’s  pa¬ 
per  on  LK  and  cut-elimination  appeared  only  four  years  later  in  1934.  However, 
we  are  reluctant  to  ascribe  much  of  the  credit  for  the  cut-elimination  theorem  to 
Herbrand  for  two  reasons:  firstly,  does  not  have  the  elegance  of  the  sequent 
calculus  LK,  and  secondly,  the  errors  in  Herbrand ^s  proof  impinge  directly  on 
the  proof  of  the  equivalence  of  Qh  and  Q'jj . 

Indeed,  it  is  precisely  at  the  step  of  “elimination  of  modus  ponens”,  which 
is  the  analogue  of  cut-elimination,  that  the  errors  in  Herbrand’s  proof  occur 
(see  paragraph  5,3,  lemma  3,  chapter  5  of  Herbrand’s  thesis).  It  is  well-known 
that  the  process  of  cut-elimination  in  first-order  logic  leads  to  superexponential 
growth  rates;  however,  in  his  erroneous  proof,  Herbrand  claimed  that  much  lower 
growth  rates  sufficed.  The  corrected  versions  of  Herbrand’s  proof,  given  by  Gbdel 
(see  [7])  and  by  Dreben  et  al.  [3,4,5]  do  give  superexponential  growth  rates  that 
are  similar  to  the  growth  rates  known  to  hold  for  the  cut-elimination  theorem; 
and  these  growth  rates  are  (nearly)  optimal. 

5.3.  The  completeness  theorem. 

Herbrand’s  thesis  also  includes  a  construction  that  is  very  close  to  the  com¬ 
pleteness  theorem.  (Recall  that  the  completeness  theorem  was  first  proved  by 
Godel  in  1930,  in  the  same  year  that  Herbrand’s  thesis  was  completed.)  In  his 
thesis,  Herbrand  discusses  that  fact  that  if  there  is  no  witnessing  substitution  for 
a  proposition  derived  from  A  (as  in  Theorem  13),  then  it  is  possible  to  construct 
an  sequence  of  finite  domains  where  appropriate  translations  of  A  are  false.  Her¬ 
brand  also  discusses  the  possibility  of  having  an  infinite  domain  where  A  would 
be  false  in  the  usual  sense;  had  he  actually  done  this,  he  would  have  proved  the 
completeness  theorem.  Somewhat  surprisingly,  Herbrand  evidently  knew  that 
such  an  infinite  domain  could  be  obtained,  but  because  of  his  constructive  out¬ 
look,  he  declined  to  carry  out  the  proof  that  such  an  infinite  domain  existed. 
Indeed  he  says 
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“but  only  a  ‘principle  of  choice’  could  lead  us  to  take  a  fixed  system  of 
values  in  an  infinite  domain  ”  ® 

By  this  he  means  that  it  would  be  necessary  to  use  the  axiom  of  choice  to  obtain 
an  infinite  model  in  which  A  is  false  under  the  usual  Tarskian  semantics. 

It  is  interesting  to  speculate  why  Herbrand  chose  not  to  state  the  complete¬ 
ness  theorem.  Firstly,  Herbrand  took  a  very  strong  constructive,  formalist  point 
of  view,  and  he  would  have  rejected  non-constructive  arguments  on  philosophi¬ 
cal  grounds.  Indeed,  Herbrand  defined  “true”  to  mean  “provable  in  Qjj”  rather 
than  “true  in  all  possible  structures” .  Secondly,  it  seems  that  Herbrand  felt  that 
his  fundamental  theorem  was  of  greater  interest  than  a  model-theoretic  com¬ 
pleteness  theorem. 

The  issue  of  the  completeness  theorem  has  also  some  bearing  on  the  status 
of  the  errors  in  Herbrand ’s  thesis.  The  errors  in  his  proof  affected  only  the 
proof-theoretic  results,  and  the  completeness  theorem,  which  Herbrand  could 
have  stated  and  proved,  would  not  have  been  affected  by  these  errors.  Therefore, 
Herbrand  could  have  obtained  a  alternative  and  correct  proof  of  his  fundamental 
theorem  by  using  the  following  argument:  suppose  >1  is  a  formula  and  there  is 
no  proposition  derived  from  A  which  has  a  witnessing  substitution;  then  by 
the  completeness  theorem,  there  is  an  infinite  domain  (i.e.,  structure)  where  A 
is  false;  therefore,  since  the  proof  system  Qh  is  sound,  there  is  no  Qj^-proof 
of  A.  This  argument  proves  the  contrapositive  of  Theorem  13  and  is  thereby  an 
error-free  proof  of  Herbrand’s  fundamental  theorem.  Of  course,  this  proof  uses 
non-constructive  methods  and  presumably  would  not  have  been  attractive  to 
Herbrand. 
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Abstract,  We  show  that  there  is  a  pair  of  disjoint  A/*P-sets,  whose  dis¬ 
jointness  is  provable  in  Sl  and  which  cannot  be  separated  by  a  set  in 
V/polyy  if  the  cryptosystem  RSA  is  secure.  Further  we  show  that  factor¬ 
ing  and  the  discrete  logarithm  are  implicitly  definable  in  any  extension 
of  52  admitting  an  AfV  -definition  of  primes  about  which  it  can  prove 
that  no  number  satisfying  the  definition  is  composite. 

As  a  corollary  we  obtain  that  the  Extended  Frege  (EF)  proof  system  does 
not  admit  a  feasible  interpolation  theorem  unless  the  RSA  cryptosystem 
is  not  secure,  and  that  an  extension  of  EF  by  tautologies  Tp  {p  primes), 
formalizing  that  p  is  not  composite,  as  additional  axioms  does  not  admit 
feasible  interpolation  theorem  unless  factoring  and  the  discrete  logarithm 
are  in  V/poly  . 


The  aim  of  this  note  is  to  present  a  pair  of  disjoint  MV  -sets,  whose  disjoint¬ 
ness  is  provable  in  Sl  and  which  cannot  be  separated  by  a  set  in  V/poly  if  the 
cryptosystem  RSA  is  secure,  and  to  implicitly  define  factoring  and  the  discrete 
logarithm  in  a  natural  V77i -extension  of  Sl  .  Such  examples  are  interesting,  since 
they  set  some  limitations  on  possible  independence  proofs  in  bounded  arithmetic 
and  lower  bound  proofs  in  propositional  calculus.  In  particular  we  will  answer  a 
question  of  [7]  about  lower  bounds  for  the  interpolation  theorem,  see  Corollary 
8,  and  a  question  of  [14]  about  such  J\fV  -pairs,  see  Corollary  9.  Furthermore 
we  will  discuss  the  unprovability  of  the  pigeonhole  principle  in  Sl . 

As  this  is  intended  to  be  a  short  technical  note,  we  shall  omit  definition  of 
most  of  the  concepts.  For  definitions  of  RSA  and  one-way  functions  see  e.g.  [10], 
a  reference  for  propositional  logic  and  bounded  arithmetic  is  [6].  The  symbol 
{a,  h)  denotes  the  greatest  common  divisor.  ZlJ-formulas  are  everywhere  in  the 
paper  meant  as  A\  w.r.t.  Sl. 

1  A  disjoint  AfV  -pair  based  on  RSA 

The  pair  is  essentially  the  well  known  probabilistic  encryption  schema  of  [1], 
which  is  known  to  be  as  secure  as  RSA,  the  most  famous  public-key  cryptosystem 
[15].  We  only  have  to  define  it  suitably,  so  that  the  disjointness  is  provable  in 
52- For  i  =  0, 1,  let 

Ai  —df  {(n, e, y);  3a:, d^r  <  n  {x  =  i  mod  2  A  a:®  =  y  mod  n 


211 


A  =  X  mod  nAy^  =  l  mod  n  A  (e,  r)  =  1}  . 

One  can  check  that  exponentiation  modulo  a  number  n  >  2  is  definable  and  sat¬ 
isfies  the  usual  relations  in  ^2,  namely  =  x^x^  mod  n,  x^^  =  {x'^Y  mod  n. 
Assuming  that  RSA  is  secure  against  an  adversary  computing  functions  in 
V/poly^  this  pair  cannot  be  separated  by  a  set  in  VIpoly  even  if  one  considers 
only  those  n’s  which  are  products  of  two  primes. 

Theorem  1. 

5*2  b  Aq  n  Ai  =0. 

In  particular^  assuming  that  RSA  is  secure  against  an  adversary  computing  func¬ 
tions  in  VIpoly,  not  all  A\-implications  provable  in  5^  admit  an  interpolant  in 
VIpoly. 

Proof. 

We  shall  show  in  Si  that  for  every  y  there  is  at  most  one  x  <  n  which 
satisfies  the  defining  formula  of  either  Aq  or  Ai .  The  proof  is  easy,  since  we  put 
everything  in  the  definition.  So  suppose  that  for  some  Xo,xi  <  n,  vq  (we  do  not 
need  ri),  do,di,  y^^  =  1  mod  n,  (e,ro)  =  1  and 

x\=y  mod  nAy^*  =  Xi  mod  n, 


for  z  =  0, 1.  We  have 

xY  =  (y^'Y^  =  =  1  inod 

for  i  =  0, 1.  Using  Euclid’s  algorithm  we  can  show  that  there  exists  an  inverse 
d!  to  e  modulo  ro-  Thus,  using  xl°  =  1  mod  n,  we  have  for  i  =  0, 1, 

y*^'  =  xY'  =  Xi  mod  n. 


whence  a^o  =  xi. 


The  statement  Aq  fi  Ai  =  0  can  be  written  as  the  following  implication 

ao{n,e\y,XQ,dQ,ro)  — »■  -’ai(n,e,i/,xi,di,ri) 

where  ai  are  ^J-formulas such  that  the  Tj-formulas  3a;, d,r  <n  ai(n,  e,y,x,  d,  r) 
define  Aj,  for  i  =  0, 1.  Any  interpolant  of  this  implication  separates  Aq  from  Ai. 
This  yields  the  second  part  of  the  theorem.  □ 

The  advantage  of  this  pair  is  that  we  do  not  have  to  mention  primes  as  is  the 
case  with  other  examples  based  on  conjectured  one-way  functions.  The  problem 
with  primes  is  that  it  is  unlikely  that  the  A/’P-definition  of  primes  of  Pratt  [12]  is 
provably  in  Si  equivalent  to  the  natural  coAfV-deUmtioii.  If  it  were  so,  we  would 
immediately  get  a  polynomial  time  algorithm  for  primality  from  Buss’s  theorem 
[2].  (It  is  unlikely  that  we  can  get  it  in  this  way.)  More  precisely,  we  would  need 
only  to  prove  that  every  number  is  either  a  (natural)  composite  number  or  a 
“Pratt  prime”. 
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The  definition  of  the  sets  Ai  we  use  is  by  no  means  unique  and  various 
modifications  are  possible.  Razborov  suggested  to  replace  =  x  mod  n  A  = 
1  mod  n  by  a;'*  =  1  mod  n.  S.Buss  pointed  out  that  by  posing  an  extra  condition 
{y,n)  ==  1  in  the  definition  we  may  drop  r  altogether  (note  that  y^  —  l(mod  n) 
implies  that  y,n  axe  coprime).  That  is  because  we  can  define  then  r  :=  d  •  e  -  1 
for  which 

yT+i  =y  (mod  n) 

This  implies,  using  the  new  condition  (y^n)  =  1,  that 

2/^  =  1  (mod  n) 

and  the  proof  of  Theorem  1  continues  a.s  before,  using  d'  :=  d. 

However,  some  condition  implying  3r{y'^  =  1  mod  n)  seems  to  be  needed,  as 
the  next  theorem  shows. 

Theorem  2.  If  RSA  is  secure,  then 

^2  !/  Vn  >  IV2/,  (2/,  n)  =  1  3r  >  0  {y^  =  1  mod  n)  . 


Proof. 

Suppose  the  converse.  Then,  again  by  Buss’s  theorem,  we  get  an  r  by  a 
polynomial  time  algorithm.  Such  an  r  suffices  to  break  the  RSA.  Namely,  if 
y  =  mod  n  and  (e,  r')  =  1,  where  P  is  the  order  of  y,  then  P\r  and  hence  also 
P\to  for  ro  =  r/{e,r).  Thus  we  have  also  (cj^o)  =  1  and  =  1  mod  n,  and  we 
compute  the  inverse  of  e  modulo  ro  and  continue  as  in  the  proof  of  Theorem  1. 
□ 


In  particular,  if  RSA  is  secure  then  S2  cannot  prove  the  Fermat-Euler  theo¬ 
rem.  Surely  it  is  possible  to  get  a  lot  of  independence  results  using  such  assump¬ 
tions,  however  the  above  sentence  is  rather  special,  since  it  easily  follows  from 
the  weak  Pigeon  Hole  Principle  (WPHP).  Thus  we  get: 

Corollary  3.  If  RSA  is  secure,  then  Si  \/  WPHP{A\). 

Proof, 

Fix  n  and  y  prime  to  each  other  and  take  the  function  r  y^  mod  n. 
By  WPHP  we  get  ri  <  7*2  <  2n  such  that  y^^  =  y^^  ^  0  mod  n,  whence 
yr2~ri  ^  jL  mod  n.  □ 


Note,  in  particular,  that  as  S2  proves  WPHP{Ai)  (by  [11])  this  gives  another 
proof  of  the  conditional  separation  Si  ^  S2  based  on  a  different  structural 
complexity  conjecture  than  the  proof  of  [9].  Moreover,  it  follows  from  [9,  4]  that 
PV  \f  WPHP{A\)  and  Si  \f  PHP{Al)  (assuming  that  NP  2  P/poly  and 
^  A2  respectively)  but  not  the  conclusion  of  Corollary  3. 
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2  Implicit  definability  of  factoring  and  the  discrete 
logarithm 

Assume  that  5^  proves 

A{x,y) /\  A{x\y) X  = 

for  some  X'J-formula  A.  In  other  words,  it  implicitly  defines  a  (partial)  function 
X  :=  f(y)  such  that 

3xA{x,y)  ^  A{f{y),y)  . 

We  show  that  in  a  natural  extension  of  5]  factoring  and  the  discrete  logarithm 
are  implicitly  definable  in  this  way. 

Let  C{p,w)  be  a  ^{-formula  saying  that 

=  {9,P,Qi,ei, . . .  ,qt,et,wi, . . .  ,wt) 

such  that: 

1.  g  e  Zp  and  ~  1  mod  p 

2.  p  -1  = 

3.  5  mod  p,  for  all  i  <t 

4.  C{qi^Wi)y  for  all  i  <t. 

The  MV  -definition  of  primes  Pratt{a)  by  Pratt  [12]  is  3w  <  t{a)C{a,  w) 
for  a  suitable  term  t{a).  Denote  by  ^  the  following  Vilf-formula: 

^  :=  Vx,  Pratt{x)  -tComposite{x) 

where  Composite{a)  :=  3u,  v  <  a,u  ‘  v  =  a. 

Theorem  4.  The  theory  A  ^  implicitly  defines  factoring,  i.e.  it  proves  the 
sequent: 

(p.q'-p'.?'  e  Pratt), p  <  q,p'  <  q' ,p  ■  q  =  a,p'  ■  q'  =  a  — >p  =  p'  A  q-q'  . 
Proof. 

We  first  show  in  Si  3-  ^  the  following 
Claim;  (PT*att(p)  A  p|a6)  — ^  (p|aVp|6). 

Assume  Pratt{p)  and  p|a6  but  that  p  does  not  divide  a.  Then  (a,p)  (which  is 
definable  in  Si  )  must  be  equal  to  1  by  ^  and  we  get  (also  in  Si  ) 

pu  3-  av  =  1 


for  some  u,  v,  and  so  also: 

pub  3-  avb  =  b  . 

By  p|a6,  p  divides  the  left-hand  side  and  hence  p\b  as  well.  This  proves  the  claim. 
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To  prove  the  theorem  let 

pq  =  p'q'  . 

Then  p\p'q^  and  by  the  claim  p|p'  or  p\q^.  But,  by  this  implies  that  p  =  p'  or 
p  —  q'.  The  same  holds  for  q  and,  by  the  assumption  p  <  q  and  p'  <q\  we  get 
p  =  p’  and  q  =  q' . 

□ 


Corollary  5.  Unless  factoring  is  in  VIpoly  ,  not  all  A\-implications  provable 
in  5]  +  ^  admit  an  interpolant  in  V/poly  . 

Proof. 

Recall  that  Pratt(a)  is  the  Z’J-formula  3w  <  t{a),  C(a,  ti;).  Denote  by  D(a,  w) 
the  ^J-formula  {w  <  t{a)  A  C{a,w))  and  by  {a)i  the  /i}-definable  function 
computing  the  bit  of  a.  By  Theorem  4  all  -implications 

D{p,u),D(q,v),p  <  q,p  ■  q  =  a,(p)i  =  1  — ► 

— ►  ^D{p\u'),-^D{q\v'),p'  >  q\p'  'q'  ^  a,  (p^i  =  1 

(and  similarly  with  {q)i  and  (q’)i  in  place  of  (p)i  and  (pOO  3-re  provable  in  52 
Interpolants  of  such  implications  compute  bits  (p)i  and  {q)i  from  a,  hence  one 
of  them  is  not  in  V/poly  unless  factoring  itself  is  in  V/poly. 

□ 


Now  we  want  a  similar  statement  for  the  discrete  logarithm. 

Theorem 6.  The  theory  5^  +  ^  implicitly  defines  the  discrete  logarithm,  i.e. 
it  proves  the  sequent: 

=  9^  niod  p,y  =  g^  mod  p,x  <  p,x'  <  p  — >  x  =  x'  . 

Proof. 

Assume  x  —  x*  =  r  >  0.  As  g^  =  p®  mod  p  we  have  g^  =  1  mod  p.  By 
C{p,  w)  also  =  1  mod  p  and  hence  also  =  1  mod  p,  where  s  =  (r,p  —  1). 
Note  that  s<p-lasr<p~l. 

For  alH  <  i  we  have  =  1;  we  could  put  this  condition  into  C(p,w) 

but  it  follows  from  the  claim  in  the  proof  of  Theorem  4  and  the  following  corollary 
of 

Claiin:  Pratt{q)  A  a|p“  — ►  a  =  p^,  some  v  <u  . 

If  a|p“  then  ah  =  q^  for  some  h  and  hence  p|o6.  By  the  claim  from  Theorem  4, 
p|a  or  p|!>.  Iterating  this  it- times  gives  a  =  q^,b  =  q^  ,  for  some  v,v^. 

Using  a  simple  property  of  gcd  available  in  ,  namely: 

(a,  u)  •  (a,  v)  =  (a,  w  •  v)  for  (w,  u)  =  1 
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we  get 

We  want  to  show  that,  for  some  j,  holds.  Assume,  for  the  sake  of 

contradiction,  that  {r,qp)  =  for  all  i  <t.  Then  by  the  above 


s  =  ni<tqp  =p-l 

which  contradicts  to  r  <  p  -  1.  So  let  j  <  i  be  a  fixed  j  such  that  (r,  q^)  <  q^ . 

Then  (r,  q^ )  —  q^\  for  some  fj  <  ej.  Here  we  use  the  above  claim  again. 

Thus  we  get 


and  so 


as  ej  ~  fj  —  1  >  0.  That  is: 


^3 

This  yields 

q^  =  =  1  mod  p 

where  u  is  the  bracket  [. . .]  in  the  equation  above. 

But  this  contradicts  the  condition  from  C{p^w): 

£Zll 

q  mod  p  . 


□ 


Similarly  to  Corollary  5  we  get 

Corollary  7.  Unless  the  discrete  logarithm  is  in  V I  poly  ,  not  all  A\-implications 
provable  in  5^  +  ^  admit  an  interpolant  in  V/poly  . 

3  Bounds  to  interpolation  theorem  for  EF 

We  shall  discuss  the  relation  of  the  above  example  to  a  question  on  interpolation 
theorems  for  propositional  proof  systems  studied  in  [7],  and  a  question  of  [14]  on 
separating  certain  pairs  of  JfV  -sets  associated  with  propositional  proof  systems. 

Let  us  rephrase  the  first  question.  Let  a  propositional  proof  system  P  be 
given.  The  problem  is  to  determine  the  complexity  of  the  function  F(c?,x)  which 
for  a  proof  d  of  a  sequent  ^(p,^)  — ►  !?^(p,f)  and  a  truth  assignment  x  for  p 

-  gives  1,  if  ^y^{x^y)  and 

-  gives  0,  if  z). 
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The  relation  to  interpolation  theorems  is  the  following.  If  we  fix  the  sequent 
and  the  proof,  then 


^{x,y)  -►  {F{d,x)  =  1)  ^{x,z). 

Thus  F{dyp)  =  1  is  like  an  interpolant.  If,  in  particular,  F  is  in  NCifpoly^  then 
we  get  an  interpolation  formula  whose  size  is  polynomial  in  the  size  of  d\  if  F  is 
in  V/poly^  then  we  get  an  interpolation  circuit  of  polynomial  size. 

Razborov  [14]  proposed  to  study  the  following  pair  of  disjoint  MV  -sets  for 
a  proof  system  P\ 

SAT*  ~df  {(6^?  is  a  satisfiable  CNF}, 

REF{P)  =df  {(0, 1*);  0  is  a  CNF  and  ->0  has  a  P-proof  of  length  t}. 

(1*  is  just  a  padding  of  length  t.)  The  problem  is:  what  can  be  the  complexity 
of  a  separating  set? 

We  observe  that  these  problems  are  essentially  equivalent.  In  fact,  their  equiv¬ 
alence  as  well  as  the  completeness  result  for  this  pair  follow  simply  from  known 
relations  between  bounded  arithmetic  and  propositional  logic  (see  [7]).  For  the 
reader  not  familiar  with  those  relations  we  sketch  a  direct  argument. 

Let  us  agree  that  we  shall  omit  the  restriction  of  being  CNF  in  the  definition 
of  SAT*  and  REF{P)  and,  instead,  let  us  take  all  formulas  in  some  complete 
basis. 

First  suppose  that  we  can  effectively  separate  SAT*  and  REF{P).  We  shall 
show  how  to  interpolate  a  sequent  >  ^{p^f)  with  a  proof  d  of  length 

t.  Let  an  assignment  a  for  p  be  given.  Consider  the  formula  ~^'F{a,f).  If  it  is 
satisfiable,  then  our  procedure  for  separating  SAT*  and  REF(P)  will  tell  us 
that  and  we  put  P(d,  a)  —  0.  If,  on  the  other  hand,  ^(a,  q)  is  satisfiable,  then 
we  get  a  proof  of  ^(a,f)  at  most  polynomially  longer  than  d,  since  we  only  need 
to  evaluate  ^(a,  q)  on  the  satisfying  assignment  and  then  apply  the  proof  of  the 
sequent.  Thus  our  procedure  will  tell  us  the  answer  also  in  this  case. 

For  the  converse  we  have  to  assume  that  the  proof  system  P  is  sufficiently 
strong,  namely,  that  it  proves  instances  of  its  own  Reflection  Principle  by  proofs 
of  polynomial  size.  This  is  true  for  EF  and  its  various  extensions;  if  stated  care¬ 
fully,  it  can  be  proved  also  for  some  (apparently)  weaker  systems,  see  [6]. 

So  assume  that  we  can  effectively  compute  an  interpolation  function  F(d,  x). 
Let  a  formula  O  of  length  n  be  given.  Consider  the  reflection  principle  for  -i0: 

Prfp^(h&lq)  ^  -6), 

where  the  propositional  variables  q  encode  proofs  of  length  t  and  ['-i0'|  is  a  truth 
assignment  which  encodes  -10.  Thus  the  sets  of  variables  of  the  formulas  in  the 
sequent  are  disjoint,  hence  F  depends  only  on  the  proof  d  of  this  sequent,  which 
is  of  polynomial  size  in  t  and  n.  If  0  is  satisfiable,  then  the  negation  of  the 
consequent  is  satisfiable,  hence  F{d)  =  0;  if  -i0  has  a  proof  of  size  <  f,  then 


217 


the  antecedent  is  satisfiable,  hence  F{d)  =  1.  Thus  we  can  effectively  separate 
SAT*  and  REF{P). 

Let  us  note,  that  the  above  argument  shows  that  the  most  difficult  sequent 
for  P,  from  the  point  of  view  of  interpolation,  is 

Prlp^iP,^)  -►  SatNeg''(p,f), 

where  SatNeg'^{p,f)  expresses  that  the  negation  of  the  formula  coded  by  p  is 
satisfied  on  the  truth  assignment  coded  by  f. 

The  interest  in  this  problem  stems  from  the  fact  that  an  upper  bound  for 
interpolation  for  P  can  yield  a  lower  bound  to  the  size  of  P-proofs.  The  idea, 
discussed  in  [5,  Sec. 5]  and  implemented  for  independence  results  for  bounded 
arithmetic  in  [13]  and  for  lower  bounds  for  propositional  logic  in  [7] ,  is  to  prove 
an  upper  bound  on  the  complexity  of  an  interpolation  function  F  for  a  proof 
system  P,  and  then  find  a  pair  of  disjoint  J\fV  -sets,  which  are  harder  to  separate 
than  it  is  to  compute  P.  The  natural  encoding  of  the  disjointness  condition  for 
the  pair  thus  gives  a  sequent  which  cannot  have  a  short  P-proof. 

On  the  other  hand,  if  such  a  sequent  does  have  a  short  P-proof  then  one  gets 
a  lower  bound  for  the  interpolation  function  for  P.  In  this  way  Theorem  1  gives: 

Corollary  8.  Assuming  that  RSA  is  secure  against  an  adversary  computing 
functions  in  V I  poly,  no  interpolation  function  for  EF  is  in  Vfpoly,  i.e.,  there 
is  a  sequence  of  sequents  which  have  polynomial  size  EF -proofs,  but  do  not  have 
polynomial  size  interpolation  circuits. 

Proof 

Let  3x,d,r  ai(n,e,y,x,d,r)  be  the  Pj-formulas  defining  Ai,  i  =  0,1,  from 
Section  1.  Since  ^2  proves 

3x,d,r  ao{n,e,y,x,d,r)  ->  -t3x,d,r  ai{n,e,y,x,d,r), 

and  it  is  a  /7i -formula,  the  translations  of  this  formula  into  propositional  calculus 
have  polynomial  size  EP-proofs  (see  [3,  8]  or  [6,  Chpt.9]).  If  RSA  is  secure,  we 
cannot  interpolate  them  by  polynomial  size  circuits.  □ 

Note  that  in  [5]  it  was  shown  that  constant  depth  Frege  systems  do  not  admit 
ATC^-bound  for  interpolation  unless  Vlpoly  C  MC^.  The  next  corollary  answers 
a  question  of  [14]. 

Corollary  9.  Assuming  that  RSA  is  secure  against  an  adversary  computing 
functions  in  a  class  C  closed  under  polynomial  time  reductions,  the  pair  of  NV 
-sets  (S AT* ,  REF{EF))  cannot  be  separated  by  a  set  in  C. 


Proof 
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This  follows  from  the  discussion  above,  or  using  Razborov’s  completeness 
result  [14]  as  follows.  Represent  the  sets  Ai  in  using  sets  encoding  the  bi¬ 
nary  representations  of  numbers.  Then,  by  RSUV-isomorphism  [16],  we  get  from 
Proposition  1: 


Since  {SAT*,  REF{EF))  is  complete  for  such  provably  disjoint  A/’^-sets,  (Ao ,  Ai) 
can  be  reduced  to  it  and  we  get  the  corollary.  □ 


4  A  problem  about  EF 

Propositions  of  section  2  show  that  it  is  quite  important  to  determine  whether  the 
formula  ^  is  provable  in  .  In  fact,  for  those  propositions  it  would  be  sufficient 
to  have  any  AfV  -definition  of  primes  (in  N)  about  which  Si  can  prove  that 
no  such  prime  is  composite.  (The  condition  (7(p,  w)  in  the  sequent  in  Theorem  6 
would  be  then  replaced  by  the  conjunction  of  the  first  three  conditions  of  C{p,  w) 
together  with  <  t{qi)  A  D{qi,Wi),  where  D  e  A\  and  3w  <  t{a)D{a,w) 

is  such  an  J\fV  -definition  of  primes.)  The  only  obstacle  to  a  proof  of  in  Si  is 
that  we  are  unable  to  prove  in  Si  a  corollary  of  Fermat-Euler  theorem: 

n  >  1  A  (2/,n)  =  1  —>■  3r  <  n,y^  =  1  mod  n  . 

To  see  this  cissume  that  p  G  Pratt  with  g  the  primitive  root  of  p  from  a  witness 
w  of  the  fact  that  p  e  Pratt.  For  the  sake  of  contradiction  assume  that  p  G 
Composite.  Then  p  is  either  a  product  p  =  ah  oi  two  coprime  a,  6  or  p  is  a  power 
p  =  a^  oi  some  a  ^  Composite  and  £>  2  (this  is  obtained  in  Si  by  looking 
at  a  decomposition  p  —  a-b  with  a  of  the  minimal  possible  length,  employing 
the  claim  from  the  proof  of  Theorem  6) .  The  corollary  of  Fermat-Euler  theorem 
would  then  imply  that  the  orders  of  g  modulo  a  and  b  are  r  <  a  and  s  <  b 
respectively.  Hence  the  order  of  g  modulo  p  is  at  most  rs  <  {a  — 1)(6  —  1)  <  p  - 1 
in  the  case  p  =  a6,  and  at  most  ra^~^  <  (a  —  <  p  —  1  in  the  case  p  =  a^. 

This  contradicts  the  assumption  that  p  is  a  primitive  root  of  p. 

Note  that  by  Theorem  2  it  is  unlikely  that  the  above  corollary  of  Fermat-Euler 
theorem  is  provable  in  Si . 

The  following  simple  theorem  shows  that  the  existence  of  such  a  formula  is 
equivalent  to  a  polynomial  upper  bound  on  the  lenghts  of  EF  proofs  of  primality. 

Theorem  10.  The  following  two  propositions  are  equivalent: 

1.  There  is  a  Sl-formula  A{a)  such  that: 

(a)  {p  G  N  I  A(p)}  is  the  set  of  primes. 

(b)  Si  \-  A{a)  -tComposite{a). 

2.  The  tautologies  t^: 

Tp  :=  1 1 -1  Composite  (o)  1 1  "'(p) 

(a  natural  propositional  translation  of  the  IIi -sentence  -i Composite (p) )  have 
polynomial  size  EF-proofs  for  all  primes  p. 


219 


Proof. 

If  2.  holds  then  the  JCj-formula  formalizing  that  Tp  has  an  EF-proof  of  size 
<  poly{\p\),  suitable  poly,  satisfies  condition  L,  as  EF  is  provably  sound  in  . 

On  the  other  hand,  having  A  satisfying  1.  construct  an  EF>proof  of  Tp  as 
follows.  Take  a  witness  to  A{p)  and  prove  that  A{p)  holds,  and  combine  this  by 
modus  ponens  with  a  polynomial  size  EF-proof  of  the  translations  of  A{p)  — > 
-\Composite{p),  getting  just  Tp.  For  details  of  the  translations  see  [8]  or  [6]. 

□ 


Hence  we  have  a  very  interesting  problem: 

Problem:  Do  the  tautologies  Tp  admit  polynomial  size  EF-proof s? 

We  do  not  know  the  answer  even  for  constant-depth  systems; 
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Frege  Proof  System  and  TNC° 


Gaisi  Takeuti* 


Frege  proof  system  F  is  any  usual  system  of  propositional  calculus  e.g.  a 
Hilbert  style  system  based  on  finitely  many  axiom  schemes  and  inference  rules. 
Extended  Frege  system  EF  is  obtained  from  F  as  follows.  An  EF-sequence  is 
a  sequence  of  formulas  . . . ,  such  that  each  'ipi  is  either  an  axiom  of  E, 
inferred  from  previous  'tpu  and  'ipv{=  ^  i^i)  hy  modes  pones  or  of  the  form 
q  (p,  where  q  is  an  atom  occurring  neither  in  (p  nor  in  any  of  . . . ,  .  Such 

q  ipjis  called  an  extension  axiom  and  q  a  new  extension  atom.  An  EE-proof 
is  any  EE-sequence  whose  last  formula  does  not  contain  any  extension  atom.  In 
[12],  S.  A.  Cook  and  R.  Reckhow  proved  that  the  pigeonhole  principle  PHP  has 
a  simple  polynomial  size  EE-proof  and  conjectured  that  PHP  does  not  admit 
polynomial  size  E-proof.  In  [4],  S.  R.  Buss  refuted  this  conjecture  by  furnishing 
polynomial  size  E-proof  for  PHP.  Since  then  the  important  separation  problem 
of  polynomial  size  E  and  polynomial  size  EF  has  not  had  any  progress. 

In  [11],  S.  A.  Cook  introduced  the  system  EV,  a  free  variable  equational  logic 
whose  provable  functional  equalities  are  ‘polynomial  time  verifiable’  and  showed 
that  metamathematics  of  E  and  EE  can  be  developed  in  PV  and  the  soundness 
of  EF  is  proved  in  PV.  In  [2],  S.  R.  Buss  introduced  the  first  order  system 
^2  and  showed  that  5^  is  essentially  conservative  extension  of  PV.  There  he 
also  introduced  a  second  order  system  Vi{BD).  In  [23]  we  proved  that  S2  and 
Vi{BD)  are  isomorphic  under  so-called  RSUV  Isomorphism.  In  [16],  J.  Krajicek 
proved  that  a  proof  in  Vi{BD)  more  precisely  Sj’^’-part  of  Vi{BD)  is  simulated 
in  polynomial  size  EE-proof  and  therefore  that  a  proof  in  (more  precisely 
S}-part  of  ^2)  can  be  simulated  by  a  polynomial  size  EE-proof. 

In  Clote-Takeuti  [10],  we  introduced  a  first  order  system  TNC°  which  cor¬ 
responds  to  computational  complexity  class  NC^ .  We  also  introduced  another 
first  order  system  T°NC°  which  is  equivalent  to  T°NC^. 

In  this  paper  we  first  develop  metamathematics  of  E  and  EE  in  TNC°  and 
prove  the  soundness  of  E  in  TNC°.  Then  we  show  that  a  proof  in  T°NC°  is 
simulated  in  polynomial  size  E  proof.  We  actually  prove  a  stronger  statement. 

Let  no  be  the  number  of  propositional  variables  in  an  E-proof  or  an  EE-proof. 
Let  /  be  an  NC^-function  defined  in  TNC°  and  TNC°  proves  that  /(a)  is  an 
E-proof  (or  an  EE-proof)  of  its  conclusion  5(a)  where  0  =  2”°.  If  we  substitute 
1,  2,  3, . . .  for  no  in  /(a),  then  /(2^),  /(2^),  /(2^), . . .  express  E-proofs  (or  EE- 
proofs)  El,  P2,  E3, . . .  .  In  this  case  we  say  “the  sequence  Ei,  E2,  E3, . . .  is  a 
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uniform  F-proof  (or  a  uniform  FF-proof).  More  precisely  /(-«)  .itself  is  called  a 
uniform  F-proof  (or  a  uniform  FF-proof).  Then  we  have  the  following  theorem. 

Theorem.  If  P  is  a  proof  of  A  in  T°NC°,  then  there  exists  a  uniform  F-proof 
of^A'^  which  is  a  natural  interpretation  of  A. 

If  F  is  a  proof  of  A  in  5^  where  A  is  an  NC^  formula  expressed  as  a  sharply 
bounded  formula  in  then  there  exists  a  uniform  FF-proof  of  ^  A~^  which 

is  a  natural  interpretation  of  A. 

Since  the  equivalence  of  an  iVC^-formula  A  and  natural  evaluation  of  natural 
interpretation  of  A  is  provable  in  TNC°,  we  have  the  following  corollary. 

Corollary,  if  TNC°  and  are  separated  by  an  NC^ -formula,  then  the  uniform 
F  and  the  uniform  EF  are  separated. 

At  the  end  we  discuss  examples  in  computatonal  complexity  namely  Razborov’s 
clique,  Raz-Wigderson’s  matching  and  Karchmer-Wigderson’s  st-connectivity 
from  our  point  of  view.  These  examples  do  not  belong  to  monotone  NC^. 
Therefore  it  is  hard  to  describe  them  in  Frege  system.  Nevertheless  their  works 
used  sort  of  implicit  definitions  of  them  which  are  stated  in  Preg  system.  We 
show  that  the  statements  used  in  the  implicit  definitions  are  polynomial  size  F 
provable.  This  might  suggest  the  possibility  that  5^  is  conservative  over  TNC° 
with  respect  to  the  NC^-formulas, 

Then  we  prove  that  the  statement  in  Karchmer-Wigdenson’s  example  has  a 
constant  depth  polynomial  size  F  proof  and  also  polynomial  size  cut-free  LK 
proof  with  substitution  but  has  no  polynomial  size  cut-free  LK  proof. 

§1.  TNC°  and  T°NC\ 

In  this  section,  we  discuss  the  properties  of  TNC°  and  T°NC°  which  we  will 
use  in  the  next  section. 

For  the  presentation  of  TNC°  and  T°NC°,  we  first  introduce  TAC°  and 
T°AC°  both  of  which  correspond  to  AC°. 

The  language  of  TAC°  and  T°AC°  consists  of  0,  1,  +,  2l^'l  •  x,  -  ,  |a;|,  a:#?/, 
MSP,  and  <,  where  the  intended  meaning  of  is  and  the 

defining  equations  of  MSP  are 

MSP{a,0)  =  0  and 

MSP{a,i-\-l)  = 

Now  let  r  be  a  theory  in  Bounded  Arithmetic.  A  formula  is  said  to  be  esb 
(essentially  sharply  bounded)  in  T  if  it  belongs  to  the  smallest  family  F  satisfying 
the  following  conditions.  See  [10]  for  the  detail. 

(1)  Every  atomic  formula  belongs  to  F. 

(2)  F  is  closed  under  Boolean  connectives. 

(3)  F  is  closed  under  sharply  bounded  quantifications. 

(4)  If  A{d,  x)  and  F(a,  x)  belong  to  F  and 

Th  <  s(a)A(a,a;)  and 


\MSP{a,i) 
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r  h  c  <  5(a),  d  <  5(a),  i4(a,  c),  A(o,  d)  ->  c  =  d 
where  c  and  d  do  not  occur  in  5(d)  and  i4(d,  x),  then  3x  <  s{d){A(d,  x)AB{d,  x)) 
and  Vx  <  5(d)(A(d, z)  3  B{d^x)  belong  to  T. 

The  theory  TAC°  is  formulated  by  the  defining  axioms  of  the  basic  functions 
and  predicates  of  the  language  and  the  following  axioms  and  inferences. 

(1)  Bit-Extensionality  Axioms. 

|a|  =  |6|,  Vz  <  la|(Bit(z, a)  =  Bit(z,  6))  — ►  a  =  6 

where  2  •  a  =  a  +  a,  mod2(a)  =  a  -  2  •  [|aj ,  and 
Bit(z,a)  =  mod2(M5P(a,  i)). 

(2)  Bit“Comprehension  Axioms 

3y  <  2l^lVz  <  |s|(Bit(z,2/)  =  l^A(z)), 
where  A{i)  is  esb. 

(3)  es^-LIND 

display where  A{a)  is  esb  and  a  satisfies  the  eigenvariable 
condition  i.e.  a  does  not  occur  in  the  lower  sequent. 

Let  T  be  a  theory  in  bounded  Arithmetic  and  A(d,  b)  be  esb  in  T.  Suppose 

T  h  V^y  <  t(x)A(x,y), 

T  h  V^2/Vz(A(f ,  y)  A  A{x,  z)  D  y  =  z) 

and  \fxA{xJ{x))  is  satisfied.  Then  T  is  said  to  e5&-defines  the  function  /. 

The  following  theorem  was  proved  in  [10]. 

Theorem  1.  A  predicate  is  in  AC°  iff  it  is  expressed  by  an  esb  formula  in 
TAC°.  A  function  is  in  AC'^  iff  it  is  esb-definable  in  TAC°. 

In  [10],  a  weak  theory  of  short  sequences  was  developed  in  TAC°.  Espe¬ 
cially  the  following  concepts  and  functions  are  expressed  by  esb  formulas  or 
es6-definable  functions  in  TAG'*. 

Seq(ii;)  :  w  is  a.  sequence  of  natural  numbers. 

P{i,w)  :  z-th  component  of  w{i  >  0). 

right(ii;)  :  |right{ii;)|  is  a  bound  for  \0{i,w)\. 

Len(ii;)  :  the  length  of  w. 

SqBd(&,  d)  :  an  upperbound  of  the  sequence  numbers  with  the  length  <  ld| 
whose  components  have  the  length  <  \b\. 

Also  fjLx  <  \t\A{x)  i.e.  the  minimum  x  <  |t|  satisfying  A(a;)  is  esft-definable  if 
A(x)  is  esb. 

Now  the  theory  T°AC^  is  obtained  from  TAG®  by  replacing  esb  formulas  by 
sharply  bounded  formulas.  Then  the  following  theorem  was  proved  in  [10]. 

Theorem  2.  Every  esb  formula  in  TAG®  is  equivalent  to  a  sharply  bounded 
formula  in  TAG®  and  TAG®  and  T°AC°  are  equivalent 

The  language  of  TNG®  is  the  same  with  the  language  of  TAG®.  TNG®  is 
obtained  from  TAG®  by  introducing  the  following  es^-BSN  (bounded  successive 
nomination) 
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b<k-^3ly<  kA{iy b, y) 
s  <k  <  SqBd(fc,  t){Seq{w)  A  right(it;)  =  k 
A  Len(if;)  =  \t\  A  /?(1,  w)  =  s 
A  Vi  <  \t\A{i,  /3{i-\-l,w)y  /3{i  -f  2,  w)), 
where  b  is  an  eigenvariable,  A;  is  a  numeral,  and  A{i,  b^y)  is  esb  in  TNC°. 

Theorem  3.  A  predicate  is  in  NC^  iff  it  is  expressed  by  an  esb  formula  in 
TNC°.  A  function  is  in  NC^  iff  it  is  esb-definable  in  TNC° 

See  [10]  for  a  proof. 

The  sum  of  bits  of  a  is  denoted  by  Count  and  has  the  following  defining 
axioms. 

Count  (0)  =  0 

a  >  0  — >  Count(a)  =  Count  (a  —  2*“^  “  ^)  +  1. 


Count  is  in  NC^  and  expressed  by  an  esb  formula  in  TNC°. 

A  term  of  the  form  \t\  is  said  to  be  small  and  a  term  of  the  form  ||t||  is  said 
to  be  very  small. 

In  [10],  we  also  introduced  a  theory  which  is  equivalent  to  TNC°. 

Here  we  use  a  slightly  different  formulation  of  such  that  the  equivalence 

of  two  versions  is  obvious. 

The  language  of  T^NC^  is  the  language  of  TAC^^  together  with  ov{x,y), 
and(a;,2/),  orand(a;o,a:i,a;2,a;3),  and  orand(a;,i). 

T°NC°  is  obtained  from  T^AC°  by  introducing  the  following  defining  axioms 
on  or(a;,2/),  and(a:,2/),  orand(xo,rri,a;2,a;3),  and  orand(a;,f). 


if  both  xq  and  xi  are  0 
otherwise . 


if  both  xo  and  xi  are  >  0 
otherwise 


orand(a:o,xi,a;2,a:3)  =  or(and(a;o,a;i),and(aj2,a;3)) 


orand(x,0)  =  orand(Bit(0,a;),Bit(l,a;),Bit(2,a:),Bit(3,a:)) 


For  a  very  small 


orand(x,i  +  1)  =orand(orand(3/o,0j  orand(2/i,i), 
or  and(2/2 ,  i)  j  or  and(y3 ,  i)) 

where 


yo  =  LSP{x,4{i  +  l)), 

2/1  =  LSP{MSP{x,  4{i  +  1)),  4{i  -f  1)), 
2/2  =  LSP{MSPix,Sii4-l))Aii  +  l)). 
2/3  =  LSP{MSP{x,  12{i  +  l)),4(f  +  1)), 
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and  LSP  satisfies  the  following  axiom 

a  =  MSP{a,  i)  •  2*  +  LSP{a,  i). 

The  intuitive  meaning  of  yo,  yi,  y2,  yz  is  expressed  by  x  =  2/0  +  2/i  •  + 

2/2(16)2(*+i)  +  2/3(16)3^*+^)  +  •  •  • . 

Theorem  4.  A  predicate  is  in  NC^  iff  it  is  expressed  by  a  sharply  bounded 
formula  in  T°NC°.  A  function  is  in  NC^  iff  it  is  definable  by  a  sharply  bounded 
formula  inT°NC°.  T^NC°  and  TNC°  are  equivalent. 

See  [10]  for  a  proof. 

i,From  now  on  by  an  NC^  formula  we  mean  either  a  sharply  bounded  formula 
in  T°NC°  or  an  esb  formula  in  TNC°. 

Similarly  by  an  NC^  function  we  mean  a  function  definable  in  a  sharply 
bounded  formula  in  r°iV(7°  or  an  esb-definable  function  in  TNC°. 

Let  A{x)  be  esb  in  TNC°.  Then  there  exists  a  unique  y  such  that 

y  <  AVi  <  |t|(Bit(i,2/)  =  1  ^(0)* 

We  define  <  tA{x)  by  Count (2/). 

Obviously  #a;  <  t(A(x)  is  es&-definable  in  TNC°. 

§2.  Metamathematics  of  F  and  EF, 

In  this  section  we  develop  metamathematics  of  F  and  EF  in  TNC°. 

We  define  i  6  a  to  be  Bit(t,  a)  =  1.  Then  the  following  comprehension  axiom 
holds. 

3a;  <  2^Vi  <  n{i  G  a;^-^^(2)), 

where  n  is  small  and  A{i)  is  an  esb  formula  without  any  occurrence  of  x. 

Our  convention  nicely  make  a  notion  of  finite  set  of  natural  numbers  but  it  has 
a  conflict  with  a  more  commonly  used  convention  to  identify  n  and  {0, 1, . . . , n- 

1}  since  we  have  to  represent  {0, 1, . . . ,  n  -  1}  by  2”  —  1. 

Therefore  we  have  to  be  very  careful  when  we  mention  a  function.  We  say  “/ 

is  a  function  defined  on  n”  if  /(O),  /(I), . . . ,  /(n  -  1)  are  defined.  In  this  case 
we  can  identify  a  function  defined  on  n  and  an  es^-definable  sequence  w  whose 
length  n  and  for  every  z  <  n,  the  z  +  1  component  /3(z  +  of  w  is  equal  to 

m- 

Definition.  A  function  /  is  the  enumeration  map  of  a  iff  /  is  defined  on 
Count  (a)  onto  a  satisfying 


Vz,  j  <  Count(a)(z  <  j<-^/(z)  <  /(j)). 
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Proposition  5.  The  enumeration  map  of  a  is  in  NC^  and  its  existence  and 
uniqueness  can  he  proved  in  TNC°. 

Proof.  Let  n  =  |a|.  First  we  prove  by  induction  on  i 

Vi  <  Count(a)3j  <  |a|(Count(L5^P(a,  j  +  1))  =  i  +  1). 

For  i  =  0,  we  can  take  j  to  be 

lij  <  |a|(Bit(aJ)  =  1). 

Suppose  i  =  io  +  1  and  Count(L5P(a,  jo  +  1))  =  +  L  Then  we  can  take  j  to 

be 

IJij  <  n{j  >  jo  A  Bit(a,  j)  =  1). 

Now  we  can  define  /(i)  by  the  following  equation 

/(i)  = /ij  <  nCount(L5P(a,  j -f  1))  =  i -f- 1.  □ 

Definition.  A  function  g  :  a  ^  bis  defined  to  be  the  pair  of  /  and  h  such  that 
/  is  the  enumeration  function  of  a  and  h  is  a  function  defined  on  n  into  6,  where 
n  is  Count  (a).  In  usual  mathematical  notation 

g{i)  = 


Definition.  A  nonempty  set  t  is  said  to  be  a  tree  if 

(1)  0  ^  t, 

(2)  E  t(|u|  >  1  D  [vl2\  E  t). 

For  a  tree  t,  one  can  show  1  E  t.  1  is  called  the  root  of  t.  Let  ui,  i;2  E  t.  We  say 
‘ui  is  the  predecessor  of  U2’  or  ‘u2  is  a  successor  of  vP  if  [U2/2J  =  vi.  Every  v 
in  a  tree  except  the  root  has  a  unique  predecessor  and  every  v  in  a  tree  has  at 
most  two  successors.  If  v  has  a  successor  u',  then  v'  must  be  either  2u  or  2u  + 1. 
Let  vi  and  V2  be  in  a  tree  t.  We  say  ‘ui  is  an  ancestor  of  V2  in  f  denoted  by 

t  t 

vi  <  V2  iff  3i  <  \v2\{vi  =  MSP{v2^i))  A  vi  ^  V2-  <  is  a  partial  order  on  We 
say  ^V2  is  a  descendant  of  Vi  in  P  if  vi  is  an  ancestor  of  V2  in  t.  If  t;  is  in  t  and 
u  7^  1,  then  1  is  ancestor  of  u. 

If  V  has  a  successor,  v  is  called  a  node.  If  v  has  no  successor,  v  is  called  a  leaf. 
For  every  node  there  exists  a  leaf  such  that  the  node  is  an  ancestor  of  the  leaf. 

Definition.  A  tree  t  is  said  to  be  left  weighted  if  for  every  node  v  of  t,  2v  is 
always  a  successor  of  u.  So  if  t  is  left  weighted  and  a  node  u  in  t  has  a  unique 
successor,  the  successor  must  be  2v. 

For  the  purpose  of  arithmetization  of  metamathematics,  we  use  2  for  -i,  3  for 
— 4  for  V,  and  5  for  A.  We  fix  a  small  number  no  and  6,  7, . . . ,  5  +  no  represent 
propositional  variables  and  6+no, . . . ,  5-:i-2no  represent  the  negation  of  variables 
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represented  by  6,  7, . . . ,  5  +  no  respectively.  We  denote  6,  7, . . . ,  5  +  no  by  poj 
and  6  +  no,...,5  +  2no  by  po,  Pi, •  •  •  ,Pno-i- 

A  formula  is  a  pair  (t,  /)  where  t  is  a  left- weighted  tree  and 

f  :  t  {0, 1, , . . ,  5  +  2no} 
satisfying  the  following  conditions. 

1.  If  u  is  a  node  in  t,  then  f{v)  is  one  of  -i,  V  and  A.  If  f{v)  is  one  of 

V,  and  A,  then  v  has  two  successors.  If  f(v)  is  -i,  then  v  has  only 
one  successor. 

2.  If  u  is  a  leaf  in  t,  then  f{v)  is  one  of  0,  1,  po,  •  •  •  ,Pno-i»  Po,  •  •  •  ,Pno-i- 

Let  t  be  a  tree  and  vet.  Then  \v\  —  1  is  called  the  height  of  v.  The  height 
of  t  is  defined  to  be  the  maximum  height  of  its  elements.  The  height  h  of  t  is 
very  small  and  satisfies  \t\  = 

A  formula  (t,  /)  is  said  to  be  a  (V,  A)-formula  if  for  every  node  v  in  f{v)  is 
either  V  or  A. 

A  (V,  A)-formula  (i,  /)  is  said  to  be  a  normal  (V,  A)-formula  if  the  following 
condition  is  satisfied. 

For  every  node  with  an  even  height  in  t,  f{v)  ~  V  and  for  every  node  with 
an  odd  height,  /(v)  =  A. 

A  normal  (V,  A)“formula  (t,  /)  is  said  to  be  complete  if  the  following  condition 
is  satisfied. 

The  height  h  of  i  is  even  and  t  is  {1,2, ..  .,2^+^  -  1}.  Le.  every  v  with 
2^  <v  <  2^+^  is  a  leaf  in  t  and  every  v  with  1  <  v  <  2^^  is  a  node  in  t. 

A  transformation  of  a  (V,  A)-formula  into  a  normal  (V,  A)-formula. 

Let  u  >  0  have  a  height  h.  For  i  <h,Vi  is  defined  by  the  following  equation 

Vi  =  MSP(v,  h  -  i). 


Let  (t,  /)  be  a  (V,  A)-formula,  v  €  t,  and  the  height  of  v  be  h.  We  define  “u 
corresponds  to  u”  by  the  following  conditions. 

1.  The  height  of  v  is  2h. 

2.  Hi  <  h  and  f{vi)  =  V  and  Vi+i  =  2vi  i.e.  Vi^i  is  the  left  successor  of 
Vi,  then  V2(i+i)  is  ^V2i  or  4v2i  +  1  i.e.  ^2(1+1)  is  a  successor  of  the  left 
successor  of  V2i 

3.  Hi  <h  and  f{vi)  =  A  and  Vi+i  =  2vi  + 1  i.e.  Vi+i  is  the  right  successor 
of  Vi,  then  ^2(1+1)  is  4i;2i  +  2  or  Av2i  +  3  i.e.  fi2(i+i)  is  a  successor  of  the 
right  successor  of  V2i. 

4.  Hi  <h  and  f{vi)  =  A  and  Vi^i  =  2vi  i.e.  Vi+i  is  the  left  successor  of  Vi, 
then  V2{i+i)  is  4i;2i  or  4v2i  +  2  i.e.  the  left  successor  of  a  successor  of  V2i- 

5.  Hi  <  h  and  f{vi)  —  A  and  Vj+i  =  2-1;*  +  1  i.e.  Uf+i  is  the  right  successor 
of  Vi,  then  ^2(1+1)  is  ^V2i  +  1  or  4^21  +  3  i.e.  the  right  successor  of  a 
successor  of  t;2t. 
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Let  us  denote  “v  corresponds  to  v”  by  A{v,v).  Then  A{v,v)  is  a  sharply 
bounded  formula.  Then  by  induction  on  the  height  of  v  h(v)  we  can  show 
V?;  6  t3vAiv). 

Since  the  height  of  t  is  very  small,  the  height  of  v  is  very  small  and  therefore 

V  is  small.  Therefore  we  can  define  if:vA{v,v)  and  show  #vA{v,v)  =  by 

the  induction  on  h(v).  We  define  v^Kv  by  <  h(v){v*  =  Vi).  Then  we  have 

v'  <v  V*  <v  and 

v'  ^  V  —y  v'  ^  V 

by  induction  on  max(h{u), /i(v'))  where  we  assume  A{v'jv')  and  A{v,v). 

We  define  i  to  be  the  set  of  all  which  satisfies  v*  =  v  ox  v*  <v  for  some  v 
which  satisfies  A(v,v)  for  some  leaf  v  in  t.  We  can  also  prove  that  for  every  v 
with  even  height  2h  in  i  there  exists  a  unique  v  with  the  height  h  in  t  such  that 

V  corresponds  to  v  and  that  A(vj  v)  implies  that  i;  is  a  leaf  in  i  iff  i;  is  a  leaf  in  i. 

Now  we  can  define  a  normal  (V,  A)-tree(t,  /)  by  defining  f{v)  for  a  leaf  v  by 
3v{f{v)  =  f  {v)  A  A{Vj  v)). 

One  can  easily  see  that  the  intended  logical  meaning  of  (t, /)  is  logically 
equivalent  to  the  intended  logical  meaning  of  (t,  /). 

A  transformation  of  a  normal  (V,  A)-formula  into  a  complete  normal 
(V,  A)-formula. 

Let  (t,  /)  be  a  (V,  A)-normal  tree  with  a  height  2h.  Then  define  t*  to  be 

and  a  complete  normal  (V,  A)-formula  (t*,/*)  is  defined  by  the  following  condi¬ 
tion. 

If  V*  is  a  leaf  of  t*  and  v  is  the  unique  leaf  in  t  satisfying  v  <  v*  V  v  =  v*, 
then 

rK)  =  /M. 

It  is  easily  seen  that  intended  logical  meaning  of  (t,  /)  is  logically  equivalent  to 
the  intended  logical  meaning  of  (t*,/*). 

A  transformation  of  a  normal  (i,  — ►)-formula  into  a  (V,  A)-formula. 

A  formula  (t,/)  is  said  to  be  (-i,  — ►)-formula  if  for  every  node  v  in  t,  f{v)  is 
either  or  A  (-i,  ~>)-formula  (t,/)  is  said  to  be  normal,  if  for  every  leaf  v 
in  t,  f{v)  is  one  of  0,  1,  po,  Pi, . . .  ,Pno-i- 

Let  (i,  /)  be  a  normal  (-»,  — ►)-formula.  It  t  does  not  have  any  node  with  two 

successors,  then  (t,  /)  must  be  of  the  form  - - ip  where  p  is  0,  1,  or  pi.  Let  h 

be  the  height  of  t.  Then  h  is  the  number  of  the  negations  in  (t,/).  If  h  is  even, 
then  we  transform  (t,  /)  to  (to,/o),  where  to  consists  solely  of  the  root  i.e.  1  and 
/o(l)  =  p.  If  h  is  odd,  then  we  transform  (t,/)  to  (to,/o)  where  to  is  the  same 
as  before  and  /o(l)  is  p  i.e.  /o(l)  is  1,  0,  or  pi  respectively  if  /(I)  is  0,  1,  or  p*. 


229 


Now  let  t  have  a  node  with  two  successors  and  %  be  the  topmost  node  with 
two  successors.  Then  for  every  u  €  t,  v  is  an  ancestor  of  vq,  v  =  vq,  or  v  is  a. 
descendant  of  vq,  since  otherwise  v  and  uq  must  meet  at  some  same  ancestor  v’ 
and  v'  must  have  two  successor. 

Let  t*  be  the  subset  of  t  which  consists  of  all  nodes  with  two  successors  and 
all  leaves.  Let  v  and  u'  be  two  members  of  t*.  We  say  “v'  is  the  left  t*-successor 
,of  v”  if  v'  is  2v  or  the  smallest  member  of  t*  which  is  a  descendant  of  2v  in  t.  We 
also  say  W  is  the  right  t*-successor  of  u” ,  if  v'  is  2v  + 1  or  the  smallest  member 
of  t*  which  is  a  descendant  of  2-1;  H- 1  in  t. 

We  say  “u  is  a  t*-leaf”  if  v  is  a  leaf  in  t.  We  say  “u  is  a  i*-node”  if  v  is  not 
a  t*-leaf.  It  is  easily  seen  that  every  t*-node  v  has  two  t* -successors.  We  define 
f*  :t*  ^  {V,A}U{0,l,po,-..,Pno-i,Po,.-.,Pno-i}  as  follows. 

Let  V  £t*  and  h  be  the  height  of  v.  If  v  is  a  i*-node  and  <  |u|(Bit(i,  v)  =  0) 
is  even  (or  odd),  then  f*(v)  =  V  (or  f*(v)  =  A).  If  v  is  a  t*-leaf,  f(v)  =  p  and 
#i  <  |u|(Bit(i,u)  =  0)  if  even  (or  odd),  then  f*(v)  =  p  (or  f*(v)  =  p).  Here  we 
make  a  convention  that  p  is  1,  0,  pi  respectively  if  p  is  0,  1,  pi. 

Now  let  vet*.  We  define  “u  corresponds  to  u''  as  follows. 

1.  If  t;  is  the  topmost  element  Uq,  then  v  is  1. 

2.  If  V  is  the  left  t*-successor  of  v*  and  corresponds  to  then  v  is  2v'. 

3.  If  V  is  the  right  t*-successor  of  and  u'  corresponds  to  v',  then  v  is 
2v'  -h  1. 

We  denote  “u  corresponds  to  v”  by  A{v^v)  for  a  while.  Then  it  is  easily 
seen  that  there  exists  a  unique  v  which  corresponds  to  v  We  define  i  to  be 
the  set  of  v  which  corresponds  to  some  member  of  t*.  We  define  f  :  i 
{0.  1,  V,  A,Po,  •  •  .  •  •  •  ,Pn„-l}  by 

m^nv) 

where  v  is  the  unique  element  of  t*  to  which  v  corresponds.  It  is  easily  seen 
that  the  intended  logical  meaning  of  (f,  /)  is  logically  equivalent  to  the  intended 
logical  meaning  of  (t,  /). 

A  transformation  of  a  normal  (-i,  V)-formula  into  a  (-«,  — ►)-formula. 

A  formula  (t, /)  is  said  to  be  (-«,  V)-formula  if  for  every  node  v  in  t,  f{v)  is 
either  or  V.  A  (-i,  V)-formula  (t,  /)  is  said  to  be  normal  if  for  every  leaf  v  in 
t,  f{v)  is  one  of  0,  1,  po,  •  •  •  ,Pno-i- 

Let  (t,  /)  be  a  normal  (->,  V)-formula  and  vet.  We  say  “u  corresponds  to  v” 
if  the  following  conditions  are  satisfied. 

1.  If  V  is  1,  then  v  is  1. 

2;Tf  v'  is  the  predecessor  of  u,  u'  has  two  successors,  v  =  2v*  (or  2v'  -|- 1), 
and  v'  corresponds  to  v\  then  v  is  (or  2u'  +  1). 

3.  If  v'  is  the  predecessor  of  v,  v'  has  a  unique  successor  v  =  2v'  and  v^ 
corresponds  to  v\  then  v  =  2v\ 

It  is  easily  seen  that  for  every  v  e  t  there  exists  a  unique  v  which  corresponds 
to  V. 
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Let  i  be  the  set  of  all  Vi,  where  Vi  is  the  i-th  subsection  of  v  and  v  corresponds 
to  some  V 

We  define  /  unique  by  the  condition  that  (t,  /)  is  a  (->,  — ►)-formula  and  f{v)  = 
f{v)iivis8i  leaf  in  t  and  v  corresponds  to  v.  It  is  easily  seen  that  the  intended 
logical  meaning  of  (f,  /)  is  logically  equivalent  to  the  intended  logical  meaning 
of(t,/). 

A  transformation  of  a  normal  (V,  A)-formula  to  a  normal  (-i,  V)-formula. 

Let  (t,  /)  be  a  normal  (V,  A)-formula  and  v  €  t.  We  say  “v  corresponds  to  v” 
if  the  following  conditions  are  satisfied. 

1.  If  V  is  1,  then  v  is  1. 

2.  If  V  is  the  left  successor  (or  the  right  successor)  ofv\  f{v^)  =  V,  f{v)  =  V, 
and  v’  corresponds  to  v\  then  v  is  the  left  successor  i.e.  2v'  (or  the  right 
successor  i.e.  2v*  +  1)  of  v'. 

3.  If  v  is  the  left  successor  (or  the  right  successor)  of  v',  /(v')  =  V,  /(v)  =  A, 
and  v’  corresponds  to  v',  then  v  is  Av’  (or  2{2v'  +  1)). 

4.  If  V  is  the  left  successor  (or  the  right  successor)  oiv\  f(v')  =  A,  f{v)  =  A, 
and  v'  corresponds  to  v\  then  v  is  the  left  successor  i.e.  2v'  (or  the  right 
successor  i.e.  2-1;'  +  1)  of  v'. 

5.  If  V  is  the  left  successor  (or  the  right  successor)  of  v\  f(v')  =  A,  f(v)  =  V, 
and  V*  corresponds  to  v',  then  v'  is  4v'  (or  2(2v'  +  1)). 

It  is  easily  seen  that  for  every  v  in  t  there  exists  a  unique  v  which  corresponds 
to  V.  We  define  t  to  be  the  set  of  all  subsections  of  v  which  corresponds  to  some 
leaf  in  t.  We  define  /  by  the  condition  that  (t,  /)  is  a  normal  (->,  V)-formula  and 
f{v)  —  f{v)  if  t)  is  a  leaf  of  t  and  v  corresponds  to  v.  It  is  easily  seen  that  the 
intended  logical  meaning  of  (t,  /)  is  logically  equivalent  to  the  intended  logical 
meaning  of  (t,  /). 

Definition.  Let  v  and  v*  have  the  height  h  and  h'  respectively.  We  define 
V  =  V'kv'  by  the  following  conditions. 

1.  The  height  of  t)  is 

2.  Vz  <  +  z,  v)  =  Bit(z,z;)) 

AVz  <  /i'(Bit(z,z})  =  Bit(z,v')) 

If  the  binary  representations  of  v  and  v*  are  lei  •  •  •  e/i  and  then  the 

binary  representation  oiv*v*  is  lei  •  •  •  c/i  c'l  •  •  •  e}^, . 

Definition.  We  denote  a  formula  (t,  /)  by  Our  intention  of  this  notation 
is  that  (t,  f)  is  a  code  for  a  formula  A,  Let  o  be  one  of  V,  A  and  —*■  and  (ti,/i) 
and  (t2,/2)  be  '"Ai"^  and  '“^2“'  respectively.  We  define  '"Ai  o  A2'^  to  be  (t,/) 
defined  by  the  following  conditions. 

1.  t  consists  of  all  numbers  of  the  form  1,  2  *  vi,  or  3  *  z;2,  where  vi  €  h 

and  V2  G  *2- 

2.  /  is  defined  by  the  following  equations 


=  /(2*vi)  =  /i(i;i)  and  /(3*t)2)  = /2(«'2). 
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Let  (to,/o)  be  We  define  ^-1^“'  to  be  (t,f)  defined  by  the  following 

conditions. 

1.  t  consists  of  all  numbers  of  the  form  1,  2  ♦  v  where  v  E  to. 

2.  /  is  defined  by  the  following  equations 

/(I)  =  -1  and  f{2  tv)  =  fo{v). 

When  we  are  dealing  with  normal  (V, A)"formulas  A,  B  or  normal  (”>,-►)- 
formulas  A,  by  '"Ao  B"'  (or  ^-lA"’  we  mean  normal  (V,  A)-formulas  or  (-1,  ->)- 
formulas  transformed  from  ""A  o  B"*. 

Let  (t,  /)  =  ''A(pii , . . . ,  be  a  formula,  where  k  is  small  but  may  not  be 
a  numeral,  and  g  be  an  es6-definable  function  in  TNC°  satisfying  that  g{py) 
is  a  formula  {tj,fj)  —  '"Bj"'.  Then  (t,/)  =  '"A(Bi, . . .  jBife)"*  is  defined  by  the 
following  conditions. 

1.  Every  leaf  in  i  is  either  a  leaf  v  of  i  satisfying  the  condition  that  f{v) 

is  none  of  pii , . . . ,  ,  or  of  the  form  v  tv'  where  v  is  a  leaf  of  t  with 

f{v)  =  Pi-  and  v'  is  a  leaf  in  tj. 

2.  If  6  i  is  not  a  leaf  satisfying  the  condition  that  f(v)  is  one  of  , . . . ,  pi^ , 

then  f{v)  =  f{v). 

3.  If  1;  G  t  is  a  leaf  satisfying  f{v)  —  pi-^  then  f{v)  =  /j(l). 

4.  liv  =  v  tv'  and  f{v)  =  pi-  and  v'  6  tj,  then  f{v)  =  fj{v'). 

Definition.  Let  g  be  an  es6-definable  function  in  TNC°  such  that  g{i)  = 
(tiifi)  =  ioT  i  <  k  where  k  is  small.  Let  h  be  the  smallest  number 

satisfying  +  1  <2^.  Then  *■  V  Bi”*  is  defined  by  the  following  (t,  /). 

i<k 

1.  ^A(poj  •  •  •  iPk-i)~'  =  (i>  /)  is  defined  as  follows. 

1.1.  t  =  {l,2,...,2'^+i  -  1}. 

1.2.  If  V  is  not  a  leaf  of  t,  then  f{v)  =  V. 

1.3.  If  V  is  2^  +  i  with  i  <  fc,  then  f{v)  =  pi. 

1.4.  If  -y  is  2^  +  i  with  fc  +  1  <  i,  then  f{v)  =  0. 

2.  (f, /)  =  '■A(Bo, .. .  jB*)"*  is  obtained  by  (t, /)  and  g  in  the  previous 

definition.  We  define  /\  Bi~'  by  replacing  V>  V,  0  in  the  definition  of 
i<k 

V  Bi~‘  by  A,  1  respectively. 

i<k 

For  the  sake  of  definiteness  we  shall  adopt  the  following  Frege  system  F.  The 
language  of  F  consists  of  0, 1,  poj  •  ■  •  jPno-i)  “'j  axioms  are  all  instances  of 

schemes:  A  — 1,  0  — ►  A,  A  ^  (B  — ►  A),  (A  — ►  (B  — ►  C))  — »•  ((A  B)  — ►  (A  — ^ 
(7)),  (^B  — >  ~»A)  — ►  (A  B),  and  its  only  rule  of  inference  is  modus  ponens: 
A  and  A  — ►  B  entail  B.  We  use  A  A  B,  A  V  B  and  A<-^B  as  abbreviations  of 
-i(A  -iB),  -iA  -►  B  and  (A  B)  A  (B  -♦  A)  respectively. 

Extended  Frege  system  EF  is  constructed  as  follows.  An  BF-sequence  is  a 
sequence  of  formulas  Aj , . . . ,  Afc  such  that  each  Aj  is  an  axiom  of  F,  inferred 
from  some  previous  A^  and  At,(=  Au  -r>  Aj)  by  modus  ponens  or  of  the  form 
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q<r^A,  where  q  is  a.  propositional  variables  (i.e.  one  of  po,  •  • .  ,i?no-i)  occurring 
neither  A  nor  any  of  ,  Ai-i.  Such  q^A  is  called  an  extension  axiom  and 
q  a  new  extension  atom.  An  EF-pvoof  is  any  EF-sequence  whose  last  formula 
does  not  contain  any  extension  atom. 

Let  whea  sequence  of  length  fc.  If  Ai , . . . ,  A*  is  an  F-proof  (or  an  EF-proof) 
and  ,  ‘"Afe"'  is  ^(1,  ly), . . . ,  /3(/u,  ty),  in  an  abuse  of  language  we  say  “w  is 

an  F-proof  (or  FF-proof).”  It  is  easily  seen  that  these  notions  are  esb  definable 
in  TNC°. 

A  formula  (t,  /)  is  called  a  sentence  if  for  every  leaf  v  in  t  f{v)  is  either  0  or 
1.  A  formula  (t,  /)  is  denoted  by  O  if  t  =  {1}  and  /(I)  =  0.  A  formula  (t,  /)  is 
denoted  by  1  if  t  =  {1}  and  /(I)  =  1. 

Definition.  A  number  a  <  2”°  is  called  an  atom  evaluation.  Let  (t,/)  be  a 
formula  of  the  form 

^A{pq,  . . .  and  a  be  an  atom  evaluation.  Then  by  S{t,  /,  a)  we  denote 

^A(€o,  •  •  • ,  where  Cj  is  O  if  Bit(i,a)  =  0  and  1  otherwise.  S{t,f,a)  is  a 

sentence.  When  (t, /)  is  '"A“’,  we  denote  S{t,f,a)  by  5'(^A“’,a). 

Let  (t,  /)  be  a  complete  normal  (V,  A)-sentence  with  height  2h  i.e.  a  complete 
normal  (V,  A)-formula  with  height  2h  which  is  a  sentence.  Then  (t,  /)  is  uniquely 

determined  by  f{2^^),  f{2^^  +  1), . .  -  1).  Let  rc  <  2^  be  defined  by 

the  formula 

where  n  =  2“^^.  Then  (t,/)  is  uniquely  determined  by  x  and  x  is  e56-definable 
from  (f, /).  We  say  “a;  expresses  (t, /).” 

Let  (t,  /)  be  complete  normal  (V,  A)-sentence  with  height  2h  and  x  express 
(t,/).  We  define  evaluation  of  (f,/)  denoted  by  eval(t,/)  as  follows 

eval(<,/)  =  /^  ,  =  ^ 

[  or  and(a;,  h  —  1)  otherwise . 

If  we  denote  (t,/)  by  ^A"*,  then  we  denote  eval(f,/)  by  eval(^A“').  Then  it  is 
easily  seen  that  the  truth  definition  defined  by  this  evaluation  is  es6-definable  in 
TNC°  and  TNC°  proves  the  usual  Tarski’s  condition.  For  a  detailed  description, 
let  (t, /)  be  a  complete  normal  (V,  A)-sentence  with  height  2h,  denoted  by  *"A"', 
and  expressed  by  x.  Then  for  h  >  0,  '"A"'  is  of  the  form  ^(Aq  A  Ai)  V  (A2  A  As)"'. 
The  numbers  xq,  xi,  X2,  X3  which  express  '"Ao"',  ^Ai"*,  ^A2'',  ^As"',  can  be 
computed  from  x  by  using  MSP  and  LSP.  Then 

evalCA"')  =  (eval('-Ao"')  A  eval('-Ai‘^))  V  (evalCAs'')  A  evalC-As^) , 

if  we  use  the  rule  1A0  =  0,  1V0  =  1,  etc.  Since  we  can  transform  any  formula 
to  a  complete  normal  (V,  A)“formula,  this  gives  an  evaluation  of  the  sentences 
which  satisfies  the  usual  Tarski’s  condition.  I.e.  evalCA  B’^)  =  eval(''A‘') 
eval(''B''),eval('’-iA‘')  =  -levalC'A’')  etc. 
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Theorem  6.  TNC°  proves  that  F  is  a  sound  proof  system  i,e. 

Atomeval(a),  Prfp{w^  /)  eval(*S'(tj  /,  a))  =  1, 

where  Atomeval(a)  and  PrfpiWyt,  f)  express  “a  is  an  atom  evaluation^*  and 
is  an  F -proof  of  a  formula  (i,  /) "  respectively. 

Proof  is  easily  done  by  induction  on  the  length  of  w. 

The  following  theorem  was  proved  by  S.  A.  Cook  in  [11],  though  he  used  PV 
in  the  place  of  . 

Theorem  7  (Cook).  proves  that  EF  is  a  sound  proof  system  i.e. 

Atomeval(a),Pr/£;ir(iji;,t,/,)  — ^  eval(5'(t,/,a))  =  1, 

where  PrfEF{w,tjf)  expresses  is  an  EF -proof  of  a  formula  (t,/)”. 

Proof.  It  is  easily  shown  by  induction  on  the  length  w  that  there  exists  an  atom 
evaluation  b  which  coincides  with  a  for  the  variables  in  (t,  /)  and  satisfies  all 
extension  axioms  used  in  w.  Then  the  theorem  is  proved  by  the  use  of  b  and 
Theorem  6. 

Remark.  In  [3],  S.  R.  Buss  proved  that  the  Boolean  formula  value  problem  is 
in  ALOGTIME  i.e.  NC^  and  complete  for  ALOGTIME  under  deterministic  log 
time  reduction  i.e.  AC°  reduction. 

Similar  contents  were  proved  with  somewhat  different  technical  methods  in 
S.  R.  Buss,  S.  A.  Cook,  A.  Gupta  and  V.  Ramachandran  [7].  A  new  and  much 
simpler  ALOGTIME  algorithm  for  the  Boolean  formula  value  problem  was  given 
in  S.  R.  Buss  [5].  An  in  [6],  S.  R.  Buss  proved  that  there  are  polynomial  size 
Frege  proofs  of  the  partial  consistency  statements  for  Frege  systems.  Therefore 
the  large  part  of  this  section  can  be  considered  as  redoing  these  in  our  setting. 

§3.  NU^  and  N^U\ 

In  [23],  we  have  introduced  the  RSUV  isomorphism.  The  basic  idea  of  the 
RSUV  isomorphism  is  that  bounded  second  order  objects  in  the  bounded  second 
order  Bounded  Arithmetic  correspond  to  first  order  objects  in  the  first  order 
Bounded  Arithmetic  and  that  first  order  objects  in  the  second  order  theory 
correspond  to  lengths  of  objects  in  the  appropriate  first  order  theory.  By  a 
bounded  second  order  object  we  mean  a  predicate  a  on  the  integers  <  a  for 
some  first  order  object  a  which  is  denoted  by  a®.  To  make  the  correspondence 
between  a  bounded  second  order  object  a®  and  a  first  order  object  6,  we  interpret 
the  truth  values  of  a®(i)  as  the  bits  in  the  binary  representation  of  6.  This 
makes  a  first  order  interpretation  of  the  second  order  concept  and  an  inverse 
second  order  interpretation  of  the  first  order  concept.  By  this  way,  we  can 
find  isomorphism  between  a  first  order  theory  and  an  appropriate  second  order 
theory.  This  isomorphism  is  called  the  RSUV  isomorphism  between  two  theories. 
In  [23],  it  is  proved  among  many  other  things  that  5^2  is  isomorphic  to  Vi{BD). 
In  [24]  it  is  also  proved  among  many  other  things  that  TAG®  is  isomorphic  to  the 
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Aj’*^(S£))-extension  of  Ti,  which  is  denoted  by  Ji,  and  that  TNC°  is  isomorphic 
to  which  is  denoted  by  NU°  in  this  paper. 

In  this  section  we  present  Ti,  Vi{BD)y  NU°,  and  N°U°  which  is  isomorphic 
to  T°NC\ 

The  language  of  Ti,  the  Aj’^(.BT))-extension  of  Ti,  consists  of 


<  . 


Notice  that  it  includes  the  multiplication  but  excludes  #. 

fi  is  a  second  order  theory  and  includes  second  order  variables  13^,..,  in 
addition  of  first  order  variables  where  t  is  a  first  order  term.  In  the  second  order 
theory,  we  use  a  notion  “abstract”.  An  abstract  is  of  the  form  {a;}A(a;),  where 
A{a)  is  a  formula  and  x  is  a  bound  variable  not  occurring  in  A{a).  An  abstract 
{a:}A(a;)  is  used  as  a  substitution  instance  for  a  free  second  order  variable  a. 
For  example  F({x}A(n:))  is  A.(0)  A  Va;(A(a:)  D  A{x  +  1))  D  A{a)  if  T’(q!)  is 
a(0)  A  Vx{a{x)  D  a{x  +  1))  D  Q:(a).  Ti  has  initial  sequents  expressing  defining 
axioms  of  function  constants  and  predicates  constants  in  the  language  and  the 
following  sequents  in  addition  of  the  initial  sequents  of  the  form  D  D 

a*(s)  s  <t 

5<ti,  s<t2,  {s) {s) 

Si  =  52,  a^{5i)  a*(52). 


The  inferences  of  Ti  are  the  following  in  addition  to  the  usual  first  order  infer¬ 
ences 

F({a;}(a:  <  t  A  A(x))),  F  A  F  A,  F({x}{x  <  t  A  A(x))) 

A  F-4  A,3^*F((/?*) 

where  A{a)  is  a  Aj’*’-formula  i.e.  a  formula  without  any  second  order  quantifiers 
or  any  first  order  unbounded  quantifiers 

F-^A,F(a*)  F{a%T-^A 

F->  3<^<F((^*),F-^  A 

where  a  satisfies  the  eigenvariable  condition  i.e.  any  second  order  variables  of 
the  form  do  not  occur  in  the  lower  sequent  Aj’*'(Bi))-IND 
A(a),F-^  A,  A(a-f  1) 

A(0),F->A,A(i) 

where  a  satisfies  the  eigenvariable  condition  and  A{a)  is  a  Aj’*'(BT))-formula. 
The  following  theorem  was  proved  in  [24]. 

Theorem  8.  TAC°  and  fi  are  isomorphic  by  the  RSUV  isomorphism. 

Vi  {BD)  is  obtained  from  Ti  by  adding  the  following  inference 
^1'\BD)  -  IND 
A(ci),  F  — ►  A,  A(a  -f- 1) 

A(0),F-.A,A(t) 

where  A{a)  is  a  S}’^(BT))“formula.  In  [23],  the  following  theorem  was  proved. 
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Theorem  9.  Si  and  Vi{BD)  are  isomorphic  by  the  RSUV  isomorphism. 

Let  T  be  a  bounded  domain  second  order  theory  in  Bounded  Arithmetic. 
Le.  all  second  order  variables  in  T  are  of  the  form  a*.  A  formula  is  said  to  be 
essentially  elementary  bounded  in  T  (abbreviated  by  eeh  in  T  or  simply  by  eeh) 
if  it  belongs  to  the  smallest  family  T  satisfying  the  following  conditions 

(1)  Every  atomic  formula  belongs  to  T. 

(2)  T  is  closed  under  Boolean  connectives. 

(3)  T  is  closed  under  bounded  quantifications. 

(4)  If  A(a®)  and  B{a^)  belong  to  T  and 

T  h  3ip^A((p^)  and 
T  h  i4(a*),  Va;  < 

where  a  and  p  do  not  occur  in  A(7*),  then  A  B{ip^))  and 

D  B{^p^))  belong  to  T. 

Now  we  are  going  to  define  NU°  which  was  denoted  by  NUl'^. 

Let  fc  be  a  natural  number.  Any  number  a  <  A:  can  be  expressed  by  a  sequence 
of  0, 1  with  the  length  |A;|  therefore  by  a  predicate  By  a  \  [a,  b]  we  denote 

{a;}(rc  <b  -  oAa^{a4-aj)).  Then  any  sequence  ao,  oi, . . .  ,a<  with  Vi  <  t{ai  <  k) 
can  be  expressed  by 

a  r  [0,  |A:1  -  1],  a  r  [|A:|,  2|fc|  -  1], . . . ,  a  t  +  m\  “  1] 

by  choosing  an  adequate  a. 

Let  /  be  defined  on  fc  4- 1  satisfying  Vi  <  k{f{i)  <  k).  If  5  <  A;,  then  we  have 
a  sequence  with  length  t 


where  f^{s)  =  f{s)  and  =  /(/”(^))-  Expressing  /  by  a  1  - 1  correspon¬ 
dence  defined  by  a  formula  and  5,  /(5), . . . ,  by 

for  a  small  t,  we  get  the  following  ee?>-BSN  (bounded  successive  nomination) 

g  <  A;  — >•  3!y  <  fcA(i,a,y) _ ' 

s  <  fc  -♦  r  [0,|fc|  -  1]  =  we®  <  Ifcl  ABit{x,s)  =  1) 

A  Vi  <  t3y  <  k{(fi  f  [i|fcl,  (i  +  l)lAi|  --  1]  =  {a:}(a:  <  |fc|  A  Bit(a:,j/)  =  1)} 

A  Vi  <  t  -  IV3/1  <  fcV3/2  <  k{>p  r  [i|*:|,  (i  +  1)|A:|  -  1]  =  {a}(a:  <  lfc|  A  Bit(®,  »i)  =  1} 
A  V  t  [(»  +  l)|fc|>(*  +  2)|A:|  -  1]  =  {a:}(«  <  |fc|  A  Bit(x,  j/2)  =  1)  3  ■^(i,  3/1, 3/2)}] 
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where  A;  is  a  positive  numeral,  a  satisfies  the  eigenvariable  condition,  A{i,  a,  h)  is 
ee6,  and 

{a:}(x  <  t  A  A{x))  <  <  5  A  B{x)) 

is  defined  to  be 

Va;  <  t{A{x)  Dx<  sAB{x))v3x  <  s[B(a:)A  >  (x  <  iAA(x))A{Vy  <  t{x  <  yAA{y))  Dy<  sAB{y)}] 


and  <tA  i4(x))  =  {x}(a:  <  5  A  B{x))  is  defined  to  be 

{x}(a:  <  tAA{x))  <  {a;}(a;  <  5  AB(a;))  A  {x}(a;  <  sAB{x))  <  {a;}(aT  <  tAA{x)). 
Now  NU^  is  obtained  from  fi  by  adding  ee6-BSN  and  the  following  inferences 

F({x}(a;<iAA(a:))),r^A  T  ^  A,F({a:}(:c  <  t  A  ^(a;))) 

r  A  r  -4  A,  3ip*F{ip*) 

where  A{x)  is  eeh  in  NU° 


A(a),r  -4  A,A{a  +  1) 

A(0),r^A,A(t) 

where  A{a)  is  eeb  in  NU^  and  a  satisfies  the  eigenvariable  condition. 

The  following  theorem  was  proved  in  [24]. 

Theorem  10.  TNC°  and  NU°  are  isomorphic  by  the  RSUV  isomorphism. 

Now  we  define  N^U°.  is  obtained  from  fi  by  introducing  third  order 

predicate  Grand  and  the  following  axioms. 

(1)  Orand(a"(“),0)<-^(Q:^(‘*)(0)  A  V  (a"(‘^)(2)  A  a"(“)(3)). 

(2)  For  small  2, 

Orand(Q:^(®),i  +  l)<-4(Ao  A  ^1)  V  {A2  A  A3) 
where 

Aq  is  Orand(Q:*(“\i), 

Ai  is  Orand({a:}(x  <  2^^  A  +  a:))), 

A2  is  Orand({a:}(a:  <  2^*  A  •  2^*  +  x))), 

and  A3  is  Orand({a:}(a:  <  2^*  A  •  2^*  +  x))). 

It  is  easily  seen  that  the  following  theorem  follows  from  [24]. 

Theorem  11.  T°NC°  and  AT®  27°  are  isomorphic  by  the  RSUV  isomorphism. 

Moreover  the  RSUV  isomorphism  implies  that  Orand(a:*(“\  i)  can  be  con¬ 
structed  as  an  eeb  formula  in  NU°  can  be  considered  as  an  elementary  bounded 
formula  in  i.e.  a  formula  without  any  second  order  quantifiers  or  any  first 

order  unbounded  quantifiers  in  AT® 27°. 
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§4.  Uniform  F  and  uniform  EF, 

First  we  simulate  the  pS{-part  of  in  uniform  EF  which  is  defined  in  the 
introduction.  A  formula  in  is  said  to  be  pure  S}  denoted  by  pSj  if  it  is 
of  the  form  3xi  <  •  •  •  3xn  <  tnA{xi , . . . ,  ajn)  where  A(xi , . . . ,  Xn)  is  sharply 

bounded.  A  sequent  Ai,...,Am  -►  is  said  to  be  a  pSj  sequent 

all  Ai, . . . , Am  are  pEj.  Si  restricted  to  pEj  sequents  is  called 

the  pEj-part  of  Si.  The  pEj-part  of  Si  is  sufficient  to  develop  the  theory  of 
polynomial  time  computational  class  P. 

We  make  similar  definitions  in  Vi{BD).  A  formula  in  Vi{BD)  is  said  to  be 
pure  eJ’^  denoted  by  pE}’*  if  it  is  of  the  form 

where  , . . . , )  is  Aq^(BD)  i.e.  elementary  bounded. 

A  sequent  Ai,...,A^  -►  in  Vi{BD)  is  said  to  be  pEj'^  if  all 

Ai , . . . ,  Am^  Bi, . . . ,  Bn  are  pEj’**.  {BD)  restricted  to  the  pEj’*^  sequents  is 
called  the  pE}’**  part  of  Vi{BD).  The  pEj’^-part  of  5^  and  the  pE}’^-part  of 
Vi{BD)  are  obviously  isomorphic  by  the  RSUV  isomorphism.  Therefore  we 
simulate  pEj’^-part  of  in  uniform  EF  in  the  place  of  simulation  of 

pEj-part  of  52- 

In  [16],  J.  Krajicek  proved  that  thepE}’^-part  of  ^^^(BB)  has  its  simulation  in 
polynomial  size  EF.  We  are  going  to  show  that  we  can  formalize  his  simulation 
using  coding  in  TNC°  and  prove  its  uniformity. 

Let  P  be  a  pEj’^  proof  in  Vi{BD).  For  simplicity,  we  assume  without  loss  of 
generality  that  the  end  sequence  of  P  has  only  one  first  order  free  variable  a. 

By  the  devise  in  pp.  378-9  in  [23],  we  can  assume  that  all  second  order 
variables  in  P  have  the  same  bound  s{a)  where  s{a)  is  a  polynomial  of  a  with 
positive  coefficients. 

Moreover  we  may  assume  without  loss  of  generality  that  P  satisfies  the  fol¬ 
lowing  normal  conditions. 

(1)  All  second  order  free  variables  are  either  in  the  end  sequent  of  P  or  used 
as  an  eigenvariable.  No  second  order  free  variables  in  the  end  sequent  of 
P  are  used  as  an  eigenvariable. 

(2)  Free  second  order  variables  used  as  an  eigenvariable  in  different  inferences 
must  be  different. 

(3)  Let  a  second  order  free  variable  a  be  used  as  an  eigenvariable  in  the 
inference  I  in  P.  Then  every  occurrence  of  a  in  P  must  be  above  I. 

We  enumerate  all  second  order  free  variables  in  P,  say  ao>  •  •  • ,  CK/t-i,  and 
also  some  second  order  bound  variables,  say  <poj  V?!  ?  •  •  •  j  where  i{a)  is  a 

polynomial  of  a.  We  define  no  =  k{s{a)  -H  1)  and  no  =  no  +  i{a){s{a)  +  1). 

We  let  no  play  the  role  of  no  in  §2  but  for  a  while  we  use  no  in  the  place  of 
no.  More  precisely  we  denote  by  Pt(s(a)+i)+j  for  j  <  s{a)  and 

by  P2no+i{8{a)-\-l)+j* 

Now  let  A  be  an  eh  (elementary  bounded)  formula  in  which  all  second  order 
free  variables  are  among  , . . . ,  . 
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We  define  (A)  by  using  formalization- of  metamathematics  of  EF.  But  we  use 
NU°  version  in  the  place  of  TNC°  version.  Therefore  no  is  not  small  now  and 
esb  will  be  replaced  by  eeb  now. 

{A)  is  a  formalized  formula  denoted  by  ^A~'  where  '"A"’  is  a  code  of  proposi¬ 
tional  formula  A. 

(A)  is  defined  in  iVC7°  as  follows. 

(1)  If  i4(ai, . . . , ttr)  is  an  atomic  formula  without  any  of  oq,  . . . , ajt-i,  then 

<A(oi,...,ar)>  is 

fix  <  l(.A(ai , . . . ,  ttr)  A  X  =  1). 

Certainly  {A{ai , . . . ,  ttr))  is  ee&-definable  in  NU°.  (A(ai, . . . ,  a^))  is  ei¬ 
ther  0  or  1  and  of  the  form  '"A"'. 

(2)  If  A  is  of  the  form  . . .  ,an)),  then  (^)  is 

fix  <  5  4-no(t(ai,...,an)  < 

where  is  5  +  i(s(a)  +  1)  +  t(ai, . . .  ,a„).  Then  (A)  is  either  0 

or  some  pr  ioi  r  <  5  A  no.  Therefore  it  is  of  the  form 

(3)  If  A  is  of  the  form  -iB,  then  (A)  i.e.  ’"A"*  is  '"-iB"' 

(4)  If  A  is  of  the  form  B  C,  then  {A)  i.e.  ’’i"'  is  C~^. 

(5)  If  A  is  of  the  form  Vx  <  t{a)B{x,a)  (or  3a:  <  <(a)B(a:,  a)),  then  (A)  i.e. 

is  y\  B{i^ay  (or  V  B{i,a)~'),  where  ''B(i,a)"'  for  t  <  t{a) 

i<t{a) 

and  A-operation  (or  V-operation)  inside  is  also  ee6-definable  in 

NU\ 

Let  A  be  of  the  form  B(ai^“^)  and  (A)  be  '’A"’.  Then  A  is  of  the  form 
B(pJ% . . .  Let  D{x)  be  elementary  bounded  and  {D{x))  be  ^B(a:)'’. 

Then  the  simultaneous  substitution  of  B(0),. . .  ,jD(s(a))  for  PoS  •  •  • 

^B{jpq\.  . .  is  ee6-definable  in  NV  and  the  result  is  denoted  by  '■B(B(0), . . .  ,  ^(5(0)))"*, 

If  C  is  of  the  form  B({a:}(a:  <  s(a)  A  D(x))),  then  we  can  prove  by  induc¬ 
tion  on  the  number  of  logical  symbols  in  B  that  (C)  i.e.  '"C"’  is  equivalent 
to  '"B(J9(0), . . . ,B(5(a)))"’  in  NU^.  From  now  on  B(pJ\ . . . is  denoted 

by  B(p“*)  and  B(B(0), . . . ,j5(s(a)))  is  denoted  by  '"B(B)''.  Now  we  define  a 
natural  atom  evaluation  {x}Bo(a;)  in  NU^  to  be 

{a;}(a:  <  k(s(a)  +  1) 

A  3i  <  k3j  <  s(a)(x  =  pj(,(o)+i)+j  A  af^^j))) 

where  3i  <  kC(i)  is  an  abbreviation  of  C(0)  V  ...  V  C(k  -  1).  Since  no  = 
k(s(a)  4- 1),  {a:}Bo(a:)  is  an  atom  evaluation  in  the  sense  that  to  each  Pi(g^a)+i)+j 
it  assigns  its  true  value  according  to 

In  §2,  a  sentence  5('”A“’,a)  is  esb-defined  in  TNC°.  By  the  RSUV  isomor¬ 
phism,  a  sentence  5(''A"’,  {a:}Bo(a:))  is  eeb-defined  in  N17°,  where  all  second 
order  free  variables  in  A  are  among  In  the  same  way,  eval  in 

§2  which  is  es6-definable  in  TNC°  is  transformed  by  the  RSUV  isomorphism  to 
Eval  which  is  ee6-definable  in  N[/°.  Then  we  have  the  following  lemma. 
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Lemma  12.  Let  A  he  an  elementary  hounded  and  all  second  order  free  variables 
in  A  he  among  . . . ,  Then  NU°  proves 

Eval{S{'-A~^,{x}Bo{x))) 

where  “"A"*  is  (A). 

Proof.  It  is  easily  proved  by  the  induction  on  the  number  of  logical  symbol  in 
A,  since  all  the  properties  of  eval  and  S  are  translated  to  the  properties  of  Eval 
and  S  by  the  RSUV  isomorphism.  □ 

Now  let  P  be  a  proof  in  pEj’**  part  of  Vi{BD).  Without  loss  of  generality  we 
may  further  assume  that  every  sequent  in  P  is  of  the  form 

where  Bi , . . . ,  (7i , . . . ,  Cv  are  elementary  bounded  and  have  no  second  order 

variables  other  than  ones  indicated  there. 

Then  we  prove  by  induction  on  the  number  of  inferences  in  P  that  there  exist 
OLi,...,OLi^  and  some  new  bound  variable  (pij , . . . , such  that 


)  A  ...  A  Buia.p"^^  ) 


Ci{n,p^\p^^^ )  V ...  V  B,{a,p^\p^^^  y 


is  ee6-definable  in  ATP®,  where  all  ao,  oiii , . . . ,  oti^  are  different,  and  ATP®  proves 
that  there  is  a  formalized  ee6-definable  EF  sequence  ending  with  this  sequent. 

Since  the  initial  sequent  is  trivial,  we  discuss  the  case  according  to  the  last 
inference  I  to  the  sequent.  We  treat  only  nontrivial  cases. 

(1)  /  is  a  pSj’^'-induction. 

Without  loss  of  generality,  we  assume  that  I  is  of  the  form 

3(pbC{bj(pb)  — ^  3(pb+iC{b-\-  l,y?fe.n) 

3(poC{0,  ipo)  -►  3(pnC{n,(pn) 

when  n  is  of  the  form  t{a).  By  the  induction  hypothesis,  there  is  an  ee6-definable 
formalized  PF-sequence  ending  with 

'■C(6,p“'>) -^(7(6  +  1, p^'>+0'’- 

Then  we  introduce  new  propositional  variables 

1  12  2  ^(a) 

Po  J  •  •  •  J  Ps{a)  J  Po  5  •  •  *  J  Ps{a) » •  •  *  j  Po  » *  •  •  >  P«(a)  * 

Then  we  have  ee6-definable  sequence 

rC(tiS)  -  l,p‘(““)  - })  ^  (5(t(2),p‘(“'))^ 
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Joining  these  implications  by  t{a)  -  1  cuts  gives  a  formalized  EF-seqnence  end¬ 
ing  with 

All  these  operations  are  ee6-definable  in  NU°. 

(2)  I  is  second  order  3  right. 

Without  loss  of  generality,  we  assume  that  I  is  of  the  form 

{a;}(x  <  s(a)  A  Z?(a;))) 

By  the  induction  hypothesis,  there  is  a  formalized  £?F-sequence  ending  with 

Then  we  introduce  a  new  propositional  variables  Po, . . .  introduce  an 

extension  axioms  for  i  =  0, . . . ,  5(a)  and  make  an  ee6-definable  EF- 

seqnence  ending  with 

(3)  I  is  the  contraction  right. 

Without  loss  of  generality,  we  assume  that  I  is  of  the  form 

By  the  induction  hypothesis,  there  exists  a  formalized  ee6-definable  EF  sequence 
ending  with 

'■B(a,p“»,p“>)<^C'(a,p“”,y)  V  C(3,p‘‘<’,p")''. 

Then  we  introduce  new  propositional  variables  pg", . . .  and  extension  ax¬ 
ioms 

P?'^(Po  A  C(a,p«°,pO)  V  (p"  A  -C(a,p«^p')), 

. . . , 

P'"(a)^iP',{a)  A  C(0,p“»,p'))  V  A 
and  construct  a  formalized  eeh-definable  EF  sequence  ending  with 
'■B(a,p“°,p“^)  (7(a,p“o,p'")'’. 

(4)  I  is  the  contraction  left. 

Without  loss  of  generality,  we  assume  that  I  is  of  the  form 

By  the  induction  hypothesis,  we  have  a  formalized  ee2>-definable  EF  sequence 
ending  with 

A  B(a,p«^p«0  C{a,p^^,pT  • 

We  substitute  p“*  for  p"j  in  the  whole  EF  sequence  and  use  contraction  and  get 
a  new  formalized  ee6-definable  EF  sequence  ending  with 

^B(a,p«^p«0->C(a,p«^pr. 
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Theorem  13,  The  construction  discussed  above  produces  a  formalized  eeb- 
definable  EF  proof  of  '"A"’  from  a  proof  of  an  elementary  bounded  formula  A 
inpj:l’'’-partofV^(BD). 

A  nice  feature  of  the  RSUV  isomorphism  is  that  the  transformation  between 
the  first  order  theory  and  the  second  order  theory  goes  automatically.  For  the 
case  of  TNC°  and  iVU®,  very  small  number,  small  number,  sharply  bounded, 
esh  etc.,  go  to  small  number,  ordinary  number,  bounded,  elementary  bounded, 
eeb  etc.  Besides  everything  goes  automatically. 

Take  for  example  Theorem  6  and  Theorem  7,  there  Atom  eval,  Prfp,  PffEF 
eval,  S  are  all  esh  in  TNC°.  Then  the  corresponding  notions  Atom  eval,  Prfp^ 
PTfEFi  Eval,  S  are  all  eeb  in  NU^  and  we  get  the  following  theorems 

Theorem  14.  NU°  proves  that  F  is  a  sound  proof  system  i.e.: 

Atom  evo?(a*W),Fr/f(;3'(“),'-A’’)  -♦  Eval{S{^ 


Theorem  15.  pE\'^-pa'rt  of  Vi{BD)  proves  that  EF  is  a  sound  proof  system 
i.e.: 

Atom  eval{a‘^'‘'>),PrfEF{0‘^'^\''A-')  ->■ 

Now  we  have  the  following  theorem. 

Theorem  16.  Let  A  be  elementary  bounded  in  Vi{BD).  Then  A  is  provable 
ofVi{BD)  iff  (A)  i.e.  ^A~'  is  provable  in  uniform  EF,  where  the 
uniformity  is  in  the  sense  of  NU° . 

Proof  If  A  is  provable  in  pSj’^-part  of  Vi{BD),  then  by  Theorem  13  is 
provable  in  uniform  EF.  If  is  provable  in  uniform  EF,  then  by  Lemma  12, 
A  is  provable  in  pS}’^-part  of  Vi{BD).  □ 

By  the  RSUV  isomorphism,  we  get  the  following  theorem. 

Theorem  17,  Let  A  be  sharply  bounded  in  Then  A  is  provable  in  52^  iff 
(A)  i.e.  ^A^  is  provable  in  uniform  EF. 

Proof.  The  theorem  is  obtained  from  Theorem  16  by  the  RSUV  isomorphism 
together  with  the  well  known  fact  that  A  is  provable  in  iff  A  is  provable  in 
uniform  pSj-part  of  S'2.  □ 

In  [9],  P.  Colte  introduced  a  free  variable  equational  logic  ALV,  whose  terms 
represent  exactly  the  NC^  computable  functions,  and  with  the  property  that  if 
“/  =  is  provable  in  ALV  then  the  infinite  family  {|/  =  p|  n,m  €  N}  of 
propositional  tautologies  admits  polynomial  length  Frege  proofs.  Relating  with 
this,  we  are  going  to  show  that  T°NC°  and  uniform  F  are  isomorphic  with 
respect  to  NC^  formulas.  Notice  that  uniform  F  proof  is  a  polynomial  size  F 
proof. 

As  is  discussed  in  §3,  and  NC^  formula  is  expressed  by  a  sharply  bounded 
formula  in  T°NC°.  A  sharply  bounded  formula  in  T'^NC^  corresponds  to  an 
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elementary  bounded  formula  in  N°U°.<  For  an  elementary  bounded  formula  A 
in  N°U°  we  have  already  defined  (>1)  i.e.  ''-4'’.  Therefore  what  we  have  to  do 
for  a  complete  definition  of  (A)  for  an  elementary  bounded  formula  in  is 

an  interpretation  of  Grand  in  NU°, 

Let  A  be  Orand{{a;}r)(x),  i),  where  i  is  small.  First  we  make  a  complete  nor¬ 
mal  (V,  A)-formula  (t,  /)  with  height  2(i  +  1)  satisfying  the  following  condition. 

For  the  leaf  with  j  f{j)  =Pj.  I.e.  Vj  < 

j)  =  Pj)’  (*)/)  uniquely  defined  and  ee&-definable  i.e.  t  and  /  are  ex¬ 
pressed  by  an  ee6-abstract  in  NU°.  We  denote  (t,  /)  by  T(po,  •  •  •  ,Pn-i)~'  where 
n  =  22(*+i).  Let  (D{a))  be  ^D{ay.  Then  {A)  is  defined  to  be 

^T{D{0),...,D{n-l)r 

i.e.  A  is  T{D{0), . .  .,D{n  —  1))  for  (A)  =  ^A~'.  Obviously  {A)  is  ee6-definable 
in  NU^  and  the  following  lemmas  are  proved  in  the  same  way  as  before. 

Lemma  18.  Let  A  be  elementary  hounded  in  N°U°.  Then  proves  the 
following  formula. 

Eval(5(^^"‘,  {a;}Bo(a;)))^A. 

Especially  if  A  is  an  axiom  on  Grand,  then  NU°  proves  Eval(i§(^>l'’,  {a:}.Bo(^)))' 

Now  we  simulate  a  proof  in  T°NC°  in  uniform  F. 

More  precisely  we  consider  a  proof  of  an  -formula  A  i.e.  a  sharply 
bounded  formula  in  T°NC^.  Then  there  is  a  proof  of  A  in  T^NC^  in  which 
every  formula  is  a  sharply  bounded  formula.  Therefore  we  consider  only  such  a 
proof.  Since  T°NC°  and  N°U°  are  isomorphic  by  the  RSUV  isomorphism,  we 
will  simulate  a  proof  in  N°U°  by  a  uniform  EF  proof.  Let  P  be  a  proof  in  N°U° 
in  which  every  formula  is  elementary  bounded.  As  before,  we  assume  that  the 
end  sequent  of  P  has  only  free  variable  a  and  all  the  second  order  variables  in 
P  are  , . . . ,  .  Let  Ai , . . . ,  Am  Bi , . . . ,  B„  be  a  sequent  in  P.  Then 

we  prove  by  induction  on  the  number  of  inferences  to  Ai, . . . ,  Am 
there  exists  a  uniform  F  proof  to  ^Ai  A  ...  A  Am  ->  Pi  V  . . .  V  Pn""  where  ^Af' 
and  ^Bj~^  are  (Ai)  and  (Bj)  respectively.  Since  the  initial  sequent  is  taken  care 
of  by  Lemma  18,  we  discuss  the  case  according  to  the  last  inference  I  to  the 
sequent.  We  treat  only  nontrivial  cases. 

(1)  I  is  Aj’^BD^IND. 

Without  loss  of  generality  we  assume  that  /  is  of  the  form 

A{a)  ■— >  A{ci  +  1) 

A(0)  -  A{t) 

where  A{a)  is  an  elementary  bounded  formula. 

Then  by  the  induction  hypothesis  there  is  a  formalized  F-proof  ending  with 


^i(a)  -4  A{a  4- 1)"". 
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Then  we  have  ee6-definable  sequence 

^A{t  -  1)  i(i)'' . 

Joining  these  implications  by  i  —  1  cuts  gives  a  formalized  F-proof  ending  with 

All  these  operations  are  ee6-definable  in  NU°, 

(2)  /  is  V  <  right. 

Without  loss  of  generality  we  assume  that  I  is  of  the  form 

a  <  t,  A  — ►  B{a) 

A-^^x  <  tB{x)  ' 

By  the  induction  hypothesis,  there  is  a  formalized  F-proof  ending  with 

t  A  A  — ►  B{ay. 

Therefore  for  i  <  t,  there  is  a  formalized  F-proof  ending  with 

'“A  ->  Bii)-^. 

As  special  cases,  we  have  formalized  F-proofs  ending  with 

•"A  B{oy,  B{iy, . . . ,  ^A  -K  B{t)-'. 

Using  A  inferences  on  these,  we  get  a  formalized  F-proof  ending  with 

t<t 

All  these  operations  are  eefr-definable  in  NU^. 

(3)  J  is  V  <  left. 

Without  loss  of  generality  we  assume  that  I  is  of  the  form 

A{t)  B 

t  <  Sj'ix  <  sA(a;)  B  ‘ 

By  the  induction  hypothesis,  there  is  a  formalized  F-proof  ending  with 

^A(t)  B-^. 

^From  this  we  can  get  a  formalized  F-proof  ending  with  ""I  A  /\  A(i)  B^ 

i<8 

if  t  <  s  and  '~0  A  A  F"’  if  s  <  t.  All  these  operations  are  ee6-definable 

i<a 

in  NU\ 
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Theorem  19.  The  construction  discussed  above  produces  a  formalized  eeb- 
definable  F -proof  of  from  a  proof  of  an  elementary  bounded  formula  A  in 


Now  we  have  the  following  theorem. 

Theorem  20.  Let  A  be  elementary  bounded  in  N°U° ,  Then  A  is  provable  in 
N°U°  iff  (^)  i.e.  ^A~'  is  provable  in  uniform  F,  where  the  uniformity  is  in  the 
sense  of  NU^. 

Proof  If  A  is  provable  in  then  there  is  an  iV°C7°-proof  of  A  in  which 

every  formula  is  elementary  bounded.  Then  by  Theorem  19,  ^A~^  is  provable 
in  uniform  F.  On  the  other  hand,  if  ^A~^  is  provable  in  uniform  F,  then  by 
Lemma  18  A  is  provable  in  NU°  and  therefore  provable  in  N°U°.  □ 

By  the  RSUV  isomorphism,  we  get  the  following  theorem. 

Theorem  20.  Let  A  be  sharply  bounded  in  T°NU°.  Then  A  is  provable  in 
T°NU°  iff  (^)  i.e.  '^A~'  is  provable  in  uniform  F. 

Proof  The  theorem  is  obtained  from  Theorem  20  by  the  RSUV  isomorphism 
together  with  the  well-known  fact  that  if  a  sharply  bounded  A  is  provable  in 
T°NC°,  then  there  exists  a  T°NC°  proof  of  A,  in  which  every  formula  is  sharply 
bound.  □ 

The  theorems  Theorem  16  and  Theorem  17  are  obtained  by  using  Lemma  12. 
Now  Lemma  18  is  a  generalization  of  Lemma  12.  By  the  use  of  Lemma  18  in 
the  place  of  Lemma  12,  we  get  the  following  Theorem  22  and  Theorem  23  from 
the  proofs  of  Theorem  16  and  Theorem  17,  Theorem  22.  Let  A  be  elementary 
bounded  in  Then  the  translation  of  A  in  pSj’^-part  of  Vi(BD)  iff  {A) 

i.e.  is  provable  in  uniform  EF,  where  the  uniformity  is  in  the  sense  of  NU^. 

Theorem  23.  Let  A  he  sharply  bounded  in  T°NU°.  Then  translation  of  A  in 
Si  is  provable  in  Si  (A)  i.e.  is  provable  in  uniform  EF. 

As  a  corollary  we  have  the  following  theorem. 

Theorem  24.  //TNC°  and  Si  are  separated  by  a  NC^  formula  A,  then  uniform 
F  and  uniform  EF  are  separated  by  (A).  Here  by  an  NC^  formula  we  mean  an 
esb  formula  in  TNC°. 

Proof  We  may  think  that  ^4  is  a  sharply  bounded  formula  in  T^NC°  and  of  the 
form  pEj  in  52-  Then  the  provability  of  {A)  in  uniform  F  is  equivalent  to  the 
probability  of  A  in  TNC°  and  the  provability  of  (A)  in  uniform  EF  is  equivalent 
to  the  probability  of  A  in  5^  -  □ 

§5.  Razborov’s  clique,  Raz-Wigderson’s  matching  and  Karchmer- 
Wigderson’s  connectivity. 

First  we  formulate  Razborov’s  clique  problem  in  [20].  For  this,  let  r  range 
over  1,  2, . . . ,  iV,  i  over  1,  2, . . , ,  n  and  j  over  1,  2, . . . ,  n  -  1.  iV  is  the  number 
of  vertices  in  the  graph  and  we  are  asking  whether  n-clique  exists  in  the  graph. 
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The  given  graph  is  described  by  edges  Anra  where  we  assume  that  n  ^  r2 
and  Arj^r2  =  ^ran-  Razborov’s  positive  test  is  an  existence  of  an  n-clique.  I.e. 
there  exist  vertices  rj , . . . ,  r Jj  which  form  a  clique.  In  order  to  express  this  we 
introduce  Bir  and  express  the  existence  of  n-clique  by  the  following  statement 

/\  ^  ^ -Rtar)  ^-lld 

A  A  ^  ■Riara  ^  ■^J’lra)  * 

iiT^ia  ri5^ra 

Razboron’s  negative  test  is  an  assignment  of  a  color  from  {1,  2, . . . ,  n  ~  1}  to  each 
vertices  such  that  there  are  no  edge  between  vertices  with  the  same  color.  In 
order  to  express  this  we  introduce  and  express  the  existence  of  such  coloring 
by  the  following  statement. 

/W/Cjv  and  /\  A  Cjr2  D  “‘■^nra)* 

r  j  riT^ra  j 

Therefore  if  a  positive  test  works,  then  any  negative  test  fails. 

This  is  expressed  by  the  following  sequent 

r->  A 


where  F  is  the  sequent  of 


r  j 

{■Riiri  A -Rtara  ^  •^T'lra^ii^^tajri^ra 

and  A  is  the  sequence  of 

{Bi^r  A  ■^iar}ii^ia,r  j  A  Cjr2  A  -^nra  }ri94r2  • 

As  is  discussed  in  the  introduction,  S.  Buss  proved  in  [4]  that  the  following  form 
of  PHP  is  polynomial  size  F  provable 


{Piij  A  Pi2j}j,iiT^i2  • 


r  -►  A  is  polysize  F  provable  since  it  is  reduced  to  PHP  by  putting  Pij  ~ 
V r  {Pir  A  Cjr  )  • 

In  Raz-Wigderson’s  example  in  [19],  iV  =  3n  is  the  number  of  vertices  of  a 
graph.  Qestion  is  whether  there  exists  a  matching  of  m  edges.  Let  r  range  over 
1,  2, . . . ,  AT,  i  over  1,  2, . . . ,  n  and  j  over  1,  2, . . . ,  n  -  1.  We  express  a  matching 
of  m  edges  of  Mir  satisfying 


A  V  and 

i  riT^ra 

/\  --VW.rAMi,,). 

r 
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Now  ^  is  a  (n  -  1)  vertices  in  the  graph  which  we  express  hy  Qjr  satisfying 

A  A  jr2) 

j  r  j  ri#r2 

and  ^  ~'^iQjir^Qj2r)- 

ji¥^h  r 

Then  we  have  a  conclusion  that  there  exists  ri  7^  r2  such  that  Vt(^iri  AM^ra 


).  I.e.  the  following  sequent  is  a  tautology. 


r,  A,  A 

where  T  is  {Vri7tr2('^in  ^  {"' Vr(^iir  A  Mi^r)}rijtr2  > 

A  is  {\/ j.Qjr}jy{~'{Qjri  A  Qjr2)}ri^r2> 

{Vr  Qhr  A  Qi2r)}ji9^i2 

and  A  is  A  Afir2)  V  ^  ^ir2)}ri^r2  • 

This  sequent  is  polysized  F  provable  as  a  special  case  of  PHP  by  putting  Pij 


\/{MirAQjr). 


In  [14]  and  [15],  Krchmer  and  Wigderson  proved  that  st  connectivity  problem 
is  not  monotone  NC^ .  The  implicit  definition  of  their  connectivity  problem  sep¬ 
arates  not  only  constant  depth  polynomial  size  F  proof  and  cut-free  polynomial 
size  LK  but  also  cut-free  polynomial  size  LK  with  substitution  and  cut-free 
polynomial  size  LK. 

Let  G  he  a,  graph  and  0,  1, . . . ,  ti  be  vertices  of  G.  We  use  the  following 
propositional  variables. 

Aij  =  Aji  for  0  <  j  <  n  and  i  ^  j. 

Bij  for  0  <  j  <  n. 

Csj  for  5  =  1,  2  and  0  <  i  <  n. 

The  intended  meaning  of  Aij  is  “(i,  j)  is  an  edge  in  G.”  We  define  two  formulas 
$  and  as  follows. 

^  /\^  V j  Bij  A  Ar<n  hi^ji^ri  A  Br-\-lj  Aij)  A  Bqq  A  Bnn- 

^  is  /\j{Cij  A  C2j)  A  /\ij{Cij  A  C2j  — ►  ~^Aij)  A  Ciq  A  C'2n* 

The  meaning  of  $  is  “0  and  n  are  connected  in  the  graph”  and  the  meaning  of 
^  is  “{0, 1,2, .  ..,n}  is  a  union  of  =  {i  |  Cu}  and  G2  ~  {i  \  0  €  Gi, 

71  G  G2,  and  for  every  i  £  Gi  and  j  G  G2,  there  are  no  edge  {i,j)  in  G.” 
Therefore  Then  $  is  provable  and  $  — ► 

is  our  candidate  to  separate  polynomial  size  F  and  polynomial  size  EF.  Here 
we  are  going  to  prove  that  $  — >  has  a  polynomial  size  EF  proof.  However 

we  use  SF  in  the  place  of  EF. 

Both  SF  and  EF  are  introduced  in  Cook-Reckhow  [12]. 

The  substitution  rule  allows  to  substitute  in  one  inference  step  simultaneously 
formulas  for  atoms: 

0(pl,...,Pn) 

C('Pl.---i'Pn)  ' 

A  Prege  system  argumented  by  the  substitution  rule  is  denoted  by  SF. 
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Reckhow  proved  in  [21]  that  any  IVege  system  polynomially  simulate  any 
other  Frege  system.  Cook,  Dowd,  and  Krajicek-Pudlak  proved  that  EF  and 
SF  polynomially  simulate  each  other.  See  Krajicek  [17]  for  a  comprehensive 
treatment  on  the  subject. 

Now  we  assume  n  =  2^.  We  define  $(s,2o,  jo)  for  s  <  d  to  be 

f\\l  ^  f\  l\  A  .Br+lj  ^ij)  ^  BoiQ  A  B2>jo. 

1<2‘  j  r<2»ijtj 

We  define  ^(io,  jo)  to  be 


f\{Cij  V  C2j)  A  /\{Cij  A  C2j  “‘^ij)  A  Cuq  a  C2jo’ 
j  ij 

The  sizes  of  $(5,zo,  jo)  and  jo)  are  polynomial  of  n. 

We  prove  by  induction  on  5  that  there  is  a  polynomial  size  SF  proof  of 


jo)  ->^(^0,  jo). 

If  s  =  0,  then  Boi^  and  Byo.  Therefore  $(s,io,  jo)  implies  Ai^j^  which  contra¬ 
dicts  ^(io,  jo). 

Now  suppose  that  $(5,io,  jo)  ""^(^Ojjo)  bas  a  polynomial  size  SF  proof 
for  s,  io,  jo.  We  are  going  to  construct  a  polynomial  size  SF  proof  of  $(s  + 
1,^0,  jo)  — ^  “‘^(^Oj  jo).  From  $(s  +  l,io,  jo),  there  exists  r  such  that  B2-r. 
Assuming  ^(to,  jo)?  either  Cir  or  C2r  holds. 

Case  1.  C2r  holds. 

In  this  case  $(s  -I-  l,io,  jo)  A  B2»r  implies  $(5,io,r).  ^(io,  jo)  A  C2r  implies 
’$f(io,r).  By  the  induction  hypothesis  $(5,io,r)  — >  -i^(io,r)  has  a  polynomial 
size  SF  proof. 

Case  2.  Cir  holds. 

We  make  a  simultaneous  substitution  of  j  for  Bij  for  alH  <  2^,  j  <  n 
in 

jo)  --^(rjjo). 

Then  we  have  $(s,r,  jo)  — ►  -■^(r,  jo).  It  is  easily  seen  that  $(5  +  Ijio?  jo)  AB2-r 
implies  $(s,r,  jo)  and  ^(io,  jo)  A  Cir  implies  ^(r,  jo).  Therefore  we  have  a 
polynomial  size  SF  proof  of 

$(s  +  l,io,  jo)  A  Cir  ->  jo). 

Let  f{s)  be  the  size  of  the  proof  of  $(5,io,  jo).  Then  the  proof  stated  above 
implies 

f{s  +  l)  <  /  W+p(n) 

where  p{n)  is  a  fixed  polynomial  of  n.  Therefore  we  get  f{x)  <  sp{n)  <  np{n). 
Therefore  we  have,  the  following  theorem. 


Theorem  25.  $  — ^  has  a  polynomial  size  EF  proof. 

Now  we  are  going  to  introduce  a  special  form  of  Gentzen’s  LK  and  prove  that 
a  formulation  of  $  — ►  in  LK  has  a  polynomial  size  cut-free  proof  in  LK  with 
substitution  which  is  denoted  by  SLK  but  it  does  not  have  a  polynomial  size 
cut-free  LK  proof. 

We  make  a  following  slight  modification  of  Gentzen’s  LK.  In  original  LK^ 
the  principal  formula  and  the  auxiliary  formula  of  the  inference  are  always  at 
the  end  of  sequent  e.g. 

r  ^  A,i4 

-nA,r^  A' 

We  generalize  every  inference-schema  so  that  the  principal  formula  and  the  aux¬ 
iliary  formula  may  not  be  at  the  end  of  the  sequent.  E.g.  the  inference  -» left  is 
now  of  the  form  „  „  ... 

r,n-^  A,A,A 

A,A* 

Therefore  we  can  eliminate  the  inference  exchange  from  LK.  We  call  this  mod¬ 
ified  LK  simply  by  LK. 

Now  we  rewrite  $  in  the  following  form 


where  Fi  is 


{\/  A  Br+lJ  D 

* 

3 


and  T2  is 


V  C2j}j  ,  {Cij  A  €2,3  D  “'Ay), Cl,!, C2, Ti¬ 


lt  is  easily  seen  from  the  previous  proof  that  Fi,  r2  has  a  polynomial  size 
cut-free  SLK  proof,  where  the  substitution  rule  can  be  written  as 

F((^l , . . . ,  </>Ti)  A(<^1 , .  .  . ,  4>n) 

Now  we  also  show  that  Fi,  F2  — ^  has  constant  depth  polynomial  size  F  proof. 
We  always  assume  Fi  and  F2  and  prove  our  statement  in  4  steps. 


1-V 


i<i  .\j<i 


Assume  its  negation  /\  /\  Bji  D  /\  -^Bik  ).  Then  we  get  by 

l<t  \i<i  J 

induction  on  i,  whence  we  get  Bn.  As  its  special  case  we  get  Bn-i  i-  This 
together  with  Bnn  implies  Aiti.  On  the  other  hand  we  get  -lAin  from  Cu  and 
C2n  which  is  a  contradiction. 
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2.  -<(721. 

By  1,  there  exists  i  >  1  such  that  A  ^  Vfc^i  Now  there  exists  fc  7^  1 

i<t 

such  that  Bik.  Now  suppose  C2i~  K  holds,  then  ~*Aik>  On  the  other  hand, 
Bi-i  1  A  Bik  implies  Aik  whence  follows  a  contradiction.  Now  assume  that  C2k 
holds.  Then  Cn  and  C2k  implies  ->Aik  whence  follows  a  contradiction  again. 

3.  f\\/{Bik)  A  -i(72fc). 

i  k 

We  prove  y{Bik)/\-^C2k)  by  induction  on  i.  H  i  =  1,  then  Bn  A -•(721  because 
k 

of  2.  Now  we  assume  BikA-^C2k^  Then  we  have  Cu.  Suppose  that  Bi^ik>  AC2k’- 
Then  k  k'  since  ->C2k  and  C2k'-  Then  Akk'  since  Bik  and  Bj+u/  and  ^Akk> 
since  Cik  and  C2k’  which  is  a  contradiction. 

4.  Now  we  show  a  contradiction. 

Take  i  to  be  n  —  1  in  V  Bik  A  ->C2k  in  3.  Then  we  have  V  Bn-ik  A  -*C2k-  Let 

k  *  1 

k  satisfy  Bn-ik  A  ->C2k’  Then  we  have  k  ^  n  since  ^C2k  and  (72n.  Then  we 
have  Akn  since  B„_ijb  and  B„n-  On  the  other  hand,  we  have  -•Afc„  since  Cik 
and  C2n-  This  is  a  contradiction. 

It  is  easily  seen  that  this  proof  can  be  converted  to  constant  depth  polynomial 
size  F  proof. 

Now  we  are  going  to  show  that  Fi,  r2  — ►  does  not  have  any  polynomial  size 
cut-free  LK  proof.  We  use  communication  complexity  in  [14]. 

Let  /  :  {Aij  \  I  <  i,  j  <  n  and  i  ^  j}  {0,1}  be  a  Boolean  formula 
expressing  that  the  vertices  1  and  n  are  connected  in  the  graph.  Then  the 
result  in  [14]  and  [15]  implies  that  the  monotone  formula  size  of  /  Lm{f)  is 
superpolynomial.  Let  Bi  G  /"^l)  and  Bq  G  Let  B  be  a  deterministic 

protocal  such  that  for  any  given  Bi ,  Bq  as  above,  two  players  I  and  II  find  by 
following  the  protocal  D  such  that  Aij  G  Bi  and  Aij  ^  Bq.  Then  Theorem  3.3.2 
in  [14]  implies  that  the  number  of  histories  of  D  is  greater  than  Lm{f)  therefore 
superpolynomial  in  n. 

Now  let  B  be  a  cut-free  LK  proof  of 

ri,r2^. 

We  are  going  to  define  a  protocal  D  from  P.  Let  Bi  and  Bq  satisfy  Bi  G 
/“^(l)  and  Bo  G  and  Gi  and  G2  be  the  graphs  expressed  by  Bi  and 

Bq  respectively.  Then  there  exists  a  path  from  1  to  n  in  (7i .  By  repeating  some 

vertices  if  necessary,  we  can  find  a  path  i  =  ii~i2~iz - in  =  n  in  G.  Then 

we  assign  true  to  Bri  iff  i  =  ir-  On  the  other  hand,  we  can  find  Ci  and  C2  for 
G2  such  that  Cl  U  C2  =  {1, 2, . . . ,  n}  and  for  any  i  G  Ci  and  j  G  C2,  C2  has  no 
edge  {ij)  and  1  G  Ci  and  n  G  C2.  Then  we  assign  true  to  Crk  ifFr  =  lAfcGCi 
or  r  =■  2  A  fc  G  C2.  The  player  I  has  the  perfect  information  on  Gi  therefore  on 
assignment  on  Bn  and  the  player  II  has  the  perfect  information  on  C2  therefore 
on  assignment  on  Crk- 

Now  we  make  the  protocal  D  as  follows.  Look  the  proof  P  from  the  bottom 
and  let  the  logical  inference  J  be  the  bottom  most  logical  inference  to  the  sequent 
considered  in  P.  We  define  the  protocal  according  to  J 


250 


Case  (1).  J  is  of  the  form 

V%i,r-^ A  V^ti2,r-4  A 
n _ _j2 _ 

VB,,i,r^A 

3 

where  \J Biji  is  {\J Bij\)  V  (V Bij2)^  In  this  case  I  checks  which  of  V and 

J  n  32  /i  ^ 

y  Bij2  is  true.  If  y  By i ,  is  true,  then  I  sends  a  message  that  he  picks  the  sequent 

j2  jfl 

y  By  1 ,  r  ^  A  if  V  Biji  is  true  or  the  sequent  V  By 2 ,  T  A  if  V  By 2  is  true. 

j2  j2 

Case  (2).  J  is  of  the  form 

■Ay  ,  r  — >  A  r  — »  A,  Bj.^i  A  Br-|.ly 
■^r,i  A  Br_|_i  J  D  Ajj- ,  r  A 

In  this  case,  I  checks  whether  B^.i  A B^+iy-  is  true  or  not.  if  B^.i  A  B^+ij  is  true, 
then  I  sends  a  message  that  he  chooses  the  sequent  Ay,  T  -4  A.  Otherwise  I 
sends  a  message  that  he  chooses  the  sequent 

r  — ►  A,  Br^i  A  Br-\-ij 

Case  (3).  J  is  of  the  form 

r-.A,B,,y  r-4  A,B,H-i.i 

r  A,  Br,i  A  Br^ij 

In  this  case  B^.i  A  must  be  false.  If  B^^f  is  false,  then  I  sends  a  message 

that  he  chooses  F  — ^  A,  Br^i.  Otherwise  B,.+iy  is  false  and  I  sends  a  message 
that  he  chooses  F  r->  A,  Br+ij. 

Case  (4) .  J  is  of  the  form 

Ciy,F^A  C2,i,F^A 
Cij  V  0*2 j,F  — ►  A 

In  this  case,  II  checks  whether  Cij  is  true  or  not.  If  Cij  is  true,  II  sends  a 
message  that  he  chooses  Ciy-,  F  — ►  A.  Otherwise  C2,j  is  true  and  II  sends  a 
message  that  he  chooses  C2J,  F  — ^  A. 

Case  (5).  J  is  of  the  form 

~»Ay ,  F  — A  F  — »  A,  Cij  A  C2,j 
^1,3  A  C2J  D  -»A*j, F  -►  A 

In  this  case,  II  checks  whether  Cij  A  C2,j  is  true  or  not.  If  Cij  A  C2J  is  true,  II 
sends  a  message  that  he  chooses  the  sequent  -lAy ,  F  ->  A.  Otherwise  II  sends 
a  message  that  he  chooses  the  sequent 


F  A,  Cij  A  C2J  . 


251 


Case  (6).  J  is  of  the  form 

r^A,(7ij  r-.A,C2,i 

r  -►  AjCij  A  (72 j 

In  this  case  (7i  j  A  (72  j  must  be  false.  If  (7i,j  is  false,  then  II  sends  a  message 
that  he  chooses  the  sequent  V  — ►  A,  Cij.  Otherwise  C2,j  must  be  false  and  II 
sends  a  message  that  he  chooses  the  sequent  F  A,  C2j- . 

Case  (7).  J  is  of  the  form 

r  "->•  A,  Ajj 
-*Aij ,  r  — >  A 

In  this  case  I  and  II  simply  go  up  to  F  — >  A,  Aij. 

Suppose  F  ^  A  comes  up  when  we  use  the  protocal  D.  If  Bij  is  in  F,  then 
Bij  must  be  true  and  if  Bij  is  in  A,  then  Bij  must  be  false.  In  the  same  way  if 
Cij  is  in  F,  then  Cij  must  be  true  and  if  Cij  is  in  A,  then  Cij  must  be  false. 

We.: assume  that  all  the  initial  sequents  are  atomic.  Then  when  D  is  termi¬ 
nated,  we  must  end  up  a  sequent  of  the  form 

Aij  — >  Aij. 

It  is  easily  seen  that  Aij  is  the  graph  Gi  but  not  in  the  graph  (72- 

The  number  of  all  histories  of  D  <  the  number  of  all  branches  in  P  <  the 
number  of  initial  sequents  in  P. 

Therefore  the  size  of  P  must  be  greater  than  Lmif)  and  therefore  superpoly¬ 
nomial. 
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Characterizing  Parallel  Time  by  Type  2  Recursions 
with  Polynomial  Output  Length 


Stephen  J.  Bellantoni 


Abstract.  Recent  work  in  recursion  theory  has  shown  that  a  significant 
number  of  traditional  complexity  classes  can  be  defined  using  a  rami¬ 
fied  form  of  recursion,  in  which  recursion  is  not  allowed  on  values  that 
themselves  were  defined  by  a  recursion.  Applications  of  this  technique 
to  parallel  complexity  classes  have  resulted  in  function  classes  that  have 
outputs  that  are  not  length-bounded  by  any  fixed  polynomial.  The  cur¬ 
rent  work  shows  that  appropriate  tiering  of  the  initial  functions  reduces 
the  output  size  to  a  polynomial;  at  the  same  time,  type  2  recursion  al¬ 
lows  one  to  access  the  results  of  preceding  computations  without  passing 
through  an  encoding. 


Traditional  recursion  theory  can  be  ramified  by  typing  the  inputs  and  outputs 
of  functions  into  “tiers”  numbered  0,  1,  2,  etc:  each  recursively-defined  function 
produces  an  output  value  one  tier  lower  than  the  recursion  variable.  Remarkably, 
this  structuring  of  recursion  gives  characterizations  of  computational  complexity 
classes  originally  defined  in  terms  of  explicit  resource  bounds.  See  Simmons  [17], 
Bellantoni  and  Cook  [1],  Leivant  [9],  [10],  [11],  Bellantoni  [3],  [2],  Leivant  and 
Marion  [12],  [13],  and  Bloch  [4]  for  characterizations. 

In  the  area  of  parallel  computational  complexity,  Bloch  [4]  characterized  the 
functions  computable  by  an  alternating  Turing  machine  in  0(log*^  n)  time  with 
no  space  bound  (“parallel  polylog  time”).  The  characterization  uses  ordinary 
type  level  1  recursion  together  with  a  pairing  function  to  code  together  the  results 
of  computations.  An  essentially  similar  characterization  was  implicit  in  Leivant’s 
subsequent  result  [10].  One  drawback  of  these  results  is  the  fact  that  the  output 
of  the  functions  is  not  polynomially  bounded.  Indeed  this  is  intrinsic  to  the 
technique  used,  which  requires  coding  together  the  results  of  subcomputations 
in  such  a  way  as  to  construct  a  computation  tree  of  depth  f2(log*'  n). 

A  type  1  characterization  of  the  parallel  class  NC  is  given  in  [2],  based  on 
Clote’s  work  [6].  A  defect  of  this  characterization  is  that  it  refers  explictly  to 
polynomials  by  including  the  #  initial  function. 

Immerman  s  well-known  results  [8]  giving  first-order  characterizations  indi¬ 
cates  that  there  is  a  connection  between  type  2  constructs  and  parallel  computa¬ 
tion.  These  non-ramified  characterizations  use  an  explicit  bound  on  the  closure 
ordinal  of  the  fixed-point  operation.  Since  this  work  characterizes  relations  rather 
than  functions,  it  is  not  intended  to  address  the  question  of  function  growth  rate. 

Ramified  second-order  constructs  were  used  by  Leivant  [11]  and  Leivant  and 
Marion  [12],  [13]  to  characterize  the  P SPACE  computable  functions  and  the 
elementary-time  computable  functions.  There  is  also  a  characterization  of  the 
parallel  polylog  time  computable  functions  implicit  in  [13].  Although  coding 
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is  not  used  to  prove  this  characterization,  the  function  outputs  still  may  be 
superpolynomial  in  length.  A  helpful  innovation  in  this  work  is  the  admission 
as  inputs  to  functionals  only  those  functions  that  have  predicative  function  type: 
the  admissible  function’s  inputs  are  of  higher  tier  than  its  output. 

The  purpose  of  the  present  work  is  to  demonstrate  techniques  for  significantly 
reducing  the  size  of  the  computed  values  without  changing  the  complexity  of 
the  computation  that  produces  each  bit  of  the  value.  Specifically,  a  ramified 
recursion  class  is  defined  such  that  all  the  functions  are  output  length-bounded 
by  a  polynomial  and,  regarded  bitwise,  the  defined  relations  are  exactly  those 
computable  by  an  alternating  Turing  machine  in  polylog  time  with  no  space 
bound. 

The  inputs  to  this  class  of  operations  are  presented  as  balanced  binary  trees. 
It  is  shown  that  if  the  inputs  are  instead  presented  as  totally  unbalanced  trees, 
then  the  induced  function  class  consists  of  the  P SPACE  computable  functions. 

A  few  technical  novelties  are  used  to  obtain  the  characterizations.  First, 
“admissible”  function  types  are  restricted  so  that  they  have  inputs  of  tier  at 
most  ,1.  At  the  same  time,  these  function  inputs  can  only  appear  when  the 
functional  is  defined  either  by  composition  or  by  higher  type  recursion  on  a  tier 
2  variable:  higher- type  recursion  is  not  allowed  on  a  tier  1  variable.  Hence,  the 
recursion  variable  is  always  of  higher  tier  than  the  inputs  to  any  of  the  parameter 
functions. 

Second,  the  tree  constructors  and  destructors  are  defined  to  have  mixed-tier 
inputs.  For  example,  where  earlier  works  allowed  the  construction  of  a  tier  0 
tree  from  two  other  tier  0  trees,  we  now  require  that  one  of  the  input  subtrees 
have  tier  1.  These  constructors,  which  are  “more  predicative”  than  the  earlier 
ones,  do  not  allow  the  construction  of  large  output  values.  The  crucial  factor  in 
controlling  output  size  is  the  type  of  the  constructors,  rather  than  the  number 
of  critical  recursion  terms  (i.e.  rather  than  the  difference  between  tree  recursion 
and  recursion  on  notation). 

■  To  carry  out  one  direction  of  the  proof,  functionals  Ej^  are  defined  that  en¬ 
capsulate  direct  recursion  over  a  tree  of  height  i?(log*'  n).  This  derived  recursion 
scheme  has  at  each  base  case  the  evaluation  of  a  function  on  a  log*'  n  length 
path  that  leads  to  a  leaf  of  the  tree.  The  base  cases  are  followed  by  step  cases 
consisting  of  the  application  of  a  step  function  at  each  interior  node  of  the  tree. 

To  carry  out  the  other  direction  of  the  proof,  we  ascribe  tiers  to  the  gates 
of  a  circuit,  including  oracle  gates.  Equivalently  we  can  assign  tiers  to  ATM 
computation  states.  To  the  author’s  knowledge,  this  is  the  first  time  ramified 
constructs  have  been  applied  directly  to  machine  states  arising  during  the  course 
of  a  computation. 

1  Background 

Consider  nonempty  binary  trees  with  leaves  labeled  with  0  or  1.  For  a  tree  u, 
define  the  length  of  u  by  |wl  =  the  number  of  leaves  in  the  tree.  The  node  height 
of  the  tree  is  \u]:  it  is  the  number  of  nodes  in  a  maximal  root-to-leaf  path  in  the 
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tree.  (This  differs  by  1  from  the  usual  definition  of  height;  note  [u]  >  1  for  all 
u.)  For  integer  m,  define  the  length  of  m  by  |m|  =  max{l,  |'log2(m  +  1)1};  thus 
lm|  >1. 

Each  tree  t  is  a  presentation  of  an  integer  value  {t}  as  follows:  (1)  if  t  has 
length  1,  then  it  denotes  the  value  0  or  1  according  as  the  leaf’s  label  is  0  or  1; 
(2)  otherwise,  t  has  a  left  subtree  u  and  a  right  subtree  v  and  the  tree  denotes 
{u}  .  2lMI  +  {v}. 

Each  integer  value  can  be  presented  as  any  one  of  infinitely  many  trees.  A 
presentation  system  P  is  an  assignment  of  a  canonical  presentation  P(m)  for  each 
integer  value  m,  together  with  a  set  of  concrete  operations  over  presentations. 
The  operations  are  described  later.  The  normal  presentation  o/  0  is  a  single  leaf 
labeled  0,  and  the  normal  presentation  o/  1  is  a  single  leaf  labeled  1.  A  binary 
presentation  system  is  normal  if  it  uses  the  normal  presentations  for  0  and  1 
and  satisfies  |P(m)|  =  \m\  for  all  integers  m;  furthermore,  for  the  system  to  be 
normal  we  require  that  #  PM- 

Let  the  presentation  system  B  (“balanced  presentation”)  be  the  normal  pre¬ 
sentation  system  in  which  m  >  1  is  presented  as  a  balanced  tree  whose  left 
child  presents  and  whose  right  child  presents  mmod2l-l”’'l/^-l .  For 

comparison,  let  the  presentation  system  S  “skewed  presentation”)  be  the  normal 
presentation  system  in  which  m  >  1  is  presented  as  a  totally  unbalanced  tree 
whose  left  child  presents  Lm/2J  and  whose  right  child  is  the  normal  presentation 
of  m  mod  2. 

A  class  of  operations  on  trees,  2r+,  is  defined  below.  Let  BI2T'^  be  the  set 
of  functions  F  :  N  such  that  there  is  an  operation  /  €  2r***, 

F{n)^{f{Bin))} 

for  all  n.  The  functions  S/2T'^  are  defined  analogously. 

Note  the  distinction  between  “function”  and  “operation”:  a  function  is  a 
mapping  on  abstract  mathematical  objects,  while  an  operation  is  a  process  per¬ 
formed  on  concrete  presentations  of  these  abstractions.  In  order  to  describe  an 
operation  one  usually  requires  an  equivalence  relation  on  physical  presentations; 
the  equivalence  classes  can  be  inferred  from  the  denotation  operator  {•}. 

The  use  of  presentations  is  not  equivalent  to  simply  computing  over  terms  of 
an  algebra,  because  the  presentation  system  always  selects  one  of  the  many  dif¬ 
ferent  possible  presentations  of  the  input  integer.  In  contrast,  computation  over 
a  term  algebra  does  not  refer  to  the  natural  numbers.  Computations  over  a  term 
algebra  cannot  be  compared  to  computations  of  Turing  machines  without  pass¬ 
ing  through  a  mapping  between  the  terms  and  the  TM  input  tapes.  Implied  in 
such  a  mapping  is  a  presentation  system  for  the  term  algebra  and  a  presentation 
system  for  the  TM. 

The  term  “computation”  refers  to  the  overall  system  —  the  computing  de¬ 
vice  plus  the  presentational  context  provided  by  the  subject.  “Computational 
complexity”  refers,  as  it  always  has,  to  the  complexity  of  the  computed  func¬ 
tion.  It  is  quite  different  from  “operational  complexity”,  the  complexity  of  the 
mechanical  operation  performed  by  the  computing  device.  For  example,  there 
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are  very  complex  operations  that  in  the  end  induce  a  constant  function  under 
every  presentation.  We  can  also  consider  “presentational  complexity” ,  the  com¬ 
plexity  of  the  subject’s  mapping  from  objects  of  attention  (abstract  integers) 
to  presentations  (concrete  trees).  A  mathematical  formalization  of  this  concept 
of  presentational  complexity  would  require  dealing  with  a  radical  difference  be¬ 
tween  the  abstract  and  the  concrete,  and  probably  can  only  be  achieved  by 
shunting  aside  philosophical  issues. 

2  The  operations  2T+ 

The  key  to  the  following  definition  of  2T‘*'  is  that  recursive  terms  can  only  appear 
in  “lower  tier”  positions  than  the  recursion  variable,  thereby  preventing  the  step 
function  from  applying  the  same  tier  of  recursion  to  the  recursive  term.  Unlike 
the  earlier  systems,  2r'''  allows  complete  access  to  a  function  defined  by  the 
preceding  recursion  step.  This  feature  requires  the  use  of  functions  as  inputs.  I 
begin  with  definitions  for  types  and  vectors. 

The  type  hierarchy  used  in  the  discussion  is  defined  in  the  usual  way,  but 
starting  with  three  ground  types  Tq,  Ti,  and  T2  each  consisting  of  the  set  of 
nonempty  binary  trees.  Level  0  types  consist  of  trees;  level  1  types  are  operations 
on  trees;  and  level  2  types  are  operationals  on  operations  and  trees.  Product 
types  are  identified  with  function  types:  for  example,  type  (cri  x  cr2)  0-3  is 
identified  with  type  cri  — ^  (cr2  — >•  0-3).  One  can  always  assume  that  the  output 
type  is  level  0,  that  is,  #0"#^  =  1-  We  often  regard  level  0  values  (trees)  as  being 
operations  with  arity  zero;  strictly  level  1  means  level  1  with  arity  greater  than 
zero.  Finally,  the  level  1  section  of  a  class  is  the  subset  consisting  of  strictly  level 
1  functions. 

Vector  notation  is  used  freely:  a  =  , . . . ,  <^n]  where  n  =  In  this  paper 

is  used  for  vector  length,  not  for  the  smash  function.  The  subscript  “o” 

removes  and  “]”;  thus  “[ao,ro]”  means  the  concatenation  of  ^  and  r.  If  p  is  a 

vector  of  functions  and  i;  is  a  vector  of  inputs,  then  (^)  means  (piU), . . . ,  (^#5^). 

In  some  places  a  semicolon  “;”  is  used  in  a  vector;  its  meaning  is  the  same  as 
«  » 

>  • 

The  tier  of  types  is  defined  by:  Tq  is  tier  0;  Ti  is  tier  1;  r2  is  tier  2;  and  the 
tier  of  a  is  the  tier  of  the  last  element,  . 

Inputs  are  restricted  to  admissible  types: 

Deiinition  1.  A  type  a  is  admissible  if  either  #cr  =  1  or  else  a  is  tier  0  and  all 
(Ti,  1  <  i  <  are  tier  1.  In  other  words,  admissible  inputs  are  trees  or  else 
operations  mapping  tier  1  trees  to  tier  0  trees. 

The  admissible  types  with  nonzero  arity  are  all  “predicative  function  types” 
in  the  sense  of  Leivant  and  Marion  [13],  because  all  the  inputs  have  tier  greater 
than  the  output.  I  further  require  that  the  function  inputs  have  lower  tier  than 
the  tier  of  the  variables  used  in  higher- type  recursion. 

The  following  “tiered  application”  conventions  will  simplify  the  notation. 
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(1)  Suppose  /  has  type  a  with  =  fc  +  1.  Usually,  refers  to 


and  using  the  notation  implies  that  the  type  of  ^  is  the  same  as  the  type  of 
the  first  input  to  /.  I  use  this  notation  in  a  more  relaxed  way.  The  expression 
refers  to 


A^l  5  •  •  •  )  1  }  Ct+1  7  •  •  ■  >  (/4l  7  •  •  •  7  1  7  ^7  ^l+l  7  •  •  •  7 

where  ai  is  the  first  type  matching  the  type  of  Using  the  notation  implies 
that  /  has  an  input  whose  type  is  the  type  of 

(2)  Furthermore,  when  appears  in  a  list,  it  separates  the  higher  tier  inputs 
on  the  left  from  the  lower  tier  inputs  on  the  right.  Thus,  fz\w\ot  is  obtained  by 
putting  the  tier  2  terms  I,  the  tier  1  terms  w,  and  the  tier  0  terms  a  into  the 
matching  inputs  of  /.  Since  all  inputs  have  admissible  type,  z  and  w  must  be 
level  0.  It  must  be  emphasized  that  the  use  of  does  not  provide  any  additional 
structure  beyond  what  is  already  present  in  a  typed  system;  it  simply  allows  one 
to  abbreviate  lengthy  notation  that  would  otherwise  be  required  for  specifying 
types. 

The  definition  of  2r+  uses  a  generalized  form  of  projection  and  composition. 
For  projections  tt,  one  may  have  for  example  an  equality  such  as  p  =  Trip  which 
the  projected  input,  p,  is  level  1  instead  of  zero.  This  is  formally  accomplished 
by  making  the  type  of  tti  include  the  arguments  to  the  projected  input:  if  p  is 
type  Ti  To,  say,  then  tti  has  type  [Ti  To,Ti,To]. 

For  compositions,  one  uses  a  generalized  form  in  which  the  the  composed 
values  may  not  be  reduced  to  level  0:  for  example,  if  h  :  (Ti  — >  To)  — ^  To  and 
p  :  T2  X  Ti  — ►  To  then  we  may  compose  them  as  f  =  \x  h{gx)  where  x  has 
type  T2  and  gx  has  type  Ti  To.  The  defined  /  has  type  T2  To.  An  instance 
of  composition  is  level  1  if  the  defined  function  is  level  1.  Together,  generalized 
projection  and  composition  can  be  used  to  define  an  apply  functional:  let  a  be 
a  level  1  type  and  let  k  —  —  1  and  r  —  [^7^17  -  •  •  7^fc]-  Define 

=  Xr,n  7ri,[^(7ri,7r,7i),  {'K2^r'r,n), . . . ,  (Tri+fc,^^  «) 

where  r  is  level  1  type  a.  The  operation  A  applies  its  first  input  to  the  second 
and  subsequent  inputs:  Ar,n  =  rn.  The  type  of  A-^  is 

Definition  2.  2T‘^  is  the  smallest  set  of  operations  with  admissible  input  types 
defined  from  the  constants  and  rules  below. 

-  General  projection:  TTi^^,  which  produces  the  ith  input,  1  <i  <  #^.  Letting 
r  =  o-f,  the  type  of  TTj,^  is  [ao,ro]. 

-  0^*  and  1^*,  for  i  G  {0,1,2}.  They  are  operations  that  produce  the  normal 
representations  of  0  and  1. 

-  for  i>  j  e  {0, 1, 2};  and  also  for  i~  j  =  2.  The  result  m  ♦  n  is  the 
tree  whose  left  child  equals  m  G  Tj  and  whose  right  child  equals  n£Ti. 
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-  C^Ti,Ti,Ti,To,Ti]  foj  ^  g  |o,  1,2},  defined  by 

{no  if  na  =  0 
Til  if  na  =  1 
na  otherwise 

-  and  for  i  €  {0, 1, 2},  defined  by 


LO  =  0  iJO  =  0 

1-1  =  1  iJl  =  1 

L(n  ♦  m)  =  n  R{n  *m)  =  m 

-  General  composition:  If  h  of  arity  k  is  in  the  class,  and  5^1, ...  are  in  the 
class,  then 

/  =  A|  h{gO 

Is  in  the  class.  Here,  if  [iri, . . . , ak,(Tk-\-i]  is  the  type  of  h  then  has  type 
ai,  possibly  a  level  1  type. 

—1  — ♦  _ 

-  Simultaneous  1-recursion:  If  h  ,  /i  ,  and  h  are  in  the  class,  then  so  is  / 

defined  by: 

fi  z;  0fW;a  =  J;  w;  a 

fiz;l,w]a  =  hj  z]w;a 

fi  z]  {u  ♦  v),w;a  —  hi  z;  (n  ♦  v),W;a,  {fz;u,w;a)j  {fz\v,w;a) 


for  all  Zj  n,  u,  ly,  and  a.  The  expressions  on  either  side  of  these  equations 
are  level  0. 

-  Simultaneous  2-recursion:  If  /i  ,  /i  ,  and  h  are  in  the  class,  then  so  is  / 
defined  by: 

fi  0,  z]  w]a  =h^  z;  w;  a 
fl  l,z]w-^a  =;  hj  z]  w\  a 

fi  {x^y),z]w;a  =  {x  ♦  y),  z;  W;  a,  {Xw  Jx,  z;  w;  a),  (Xw  Jy,  z;  w;  a) 

for  all  X,  y,  z,  w,  and  a.  The  expressions  on  either  side  of  these  equations 
are  level  0. 

Rather  than  include  a  general  rule  of  flat  recurrence,  I  have  just  included  the 
three  important  instances  of  the  rule,  namely  C,  X,  and  R. 

Tier  2  terms  can  effectively  be  substituted  into  tier  1,  and  tier  1  terms  can 
be  substituted  into  tier  0,  using  an  operation  defined  by  composing  R  and  *. 
In  [10],  K  was  defined  by  recursion. 

In  a  system  with  infinitely  many  tiers,  one  should  be  able  to  admit  recursion 
on  all  predicative  function  types  and  have  *  at  each  tier  having  only  one  input 
from  that  tier. 


3  Bounding  2T+ 

The  first  goal  is  to  bound  the  output  length  of  the  functions  by  a  polynomial. 

For  a  tree  u,  the  length  |itl  >  1  is  the  number  of  leaves  in  w,  and  the  node 
height  M  >  1  is  the  number  of  nodes  in  a  maximal  root-to-leaf  path.  These  can 
be  generalized  to  functions  by  looking  at  the  maximum  output  length. 

Definition  3.  For  an  operation  g  of  admissible  type,  define  a  monotone  function 
\9\  by:  \g\j,k  =  mdix.-^\g]W]l  where  the  max  is  taken  over  {w  :  \wi\  < 

j  A  Truil  <  k}.  Similarly,  define  monotone  \g]  by  where 

the  max  is  taken  over  the  same  values  of  w. 

In  this  notation,  \f\j,  k  means  {\ri  | j.  A;), . . . ,  (|r^r|ij  k). 

The  degree  of  the  bounding  polynomials  is  directly  related  to  the  nesting 
depth  of  recursion.  Clote’s  definition  of  “rank”  [6]  can  be  modified  to  formalize 
the  nesting  depth  of  recursion: 

Definition  4.  The  8-rank  of  /  is  and  the  1-rank  is  p(,  defined  by: 


—  If  /  €  2T^  is  one  of  the  initial  functions,  then  =  p{  =  0. 

—  If  /  is  defined  by  composition  from  h  and  p,  then  p2  =  naax{p2  5P2} 

Pi  =max{pj,p?}. 

—  If  /  is  defined  by  simultaneous  2-recursion,  then  P2  =  max{l  +  P2*  ?  P2* » P2*  }| 

j  f  ( 1 

and  ^ 

—  If  /  is  defined  by  simultaneous  1-recursion,  then  p^  =  max{p2‘  ,P2*  jp2* }/ 
and  pj  =  max{l -I- PiSpiSPi' If. 


In  many  cases  we  will  only  be  interested  in  recursions  whose  depth  is  not  a 
constant.  By  examining  the  derivation  of  /  one  can  determine  whether  or  ^ot .the 
recursion  variable  of  a  particular  recursion  has  been  replaced  by  a  constant  ~ 
that  is,  by  composition  with  a  zero-ary  function.  I  call  these  constant  recursions. 
Define  p^  and  p(  similarly  to  p^  and  p(,  except  that  if  f  is  defined  by  a 

constant  2-recursion  or  constant  1-recursion  then  P2  =  max{p2*  >P2SP2*  }/ 

ys/  r  ys/l?  y\/l|  1 

p(  =max{pi*,pi',pi'},. 


Definition  5.  Let  be  the  set  of  /  G  2r+  such  that  =  k  and  p{  =  c. 


Clearly,  if  Jb'  <  Jfc  and  c'  <  c  then  2T^^^,  C  because  one  can  always  add 

extra  recursions  that  have  no  effect  on  the  computed  function. 

The  definition  of  “nicely  bounded” ,  given  below,  is  designed  to  suit  the  struc¬ 
ture  of  2T+.  To  get  a  rough  idea  of  it,  let’s  ignore  various  features  of  2T+  such 
as  the  conditional  function.  In  each  2T+  recursion,  we  build  up  a  lopsided  tree 
of  height  log  n.  The  deepest  leaf  of  this  tree  is  replaced  with  a  tier  0  subtree.  The 
other  leaves  are  replaced  by  tier  1  or  2  subtrees.  These  higher-tier  subtrees  are 
added  on  by  repeating  logn  applications  of  *;  therefore  they  appear  at  depths 


260 


decreasing  from  logn  through  1.  Each  time  we  perform  such  a  recursion,  the  con¬ 
structed  tree  is  tier  0,  so  outer  nested  recursions  can  only  use  this  constructed 
tree  once,  as  the  deepest  part  of  a  larger  tree.  Overall,  a  typical  constructed 
value  is  a  lopsided  tree  in  which  a  subtree  of  a  tier  0  input  appears  at  depth  at 
most  0(log  n)  and  in  which  there  are  O(log^n)  tier  1  or  2  subtrees  appearing 
at  shallower  and  shallower  depths. 

Definition  6.  An  operation  Xz\w]a,r  fz;w;a,f  in  2r+  is  nicely  hounded  if 
there  is  a  constant  c/  >  1  such  that  for  all  values  w\ a,  f : 

-  {8T2)  If  /  is  tier  2, 

|/z;w;a,f|  <  c/ •  max{l,  |z|} 

<  max{fj]} -hc/ 

-  (STi)  If  /  is  tier  1, 

\fz;w;a,f\  <  c/ •  max{l,  |:2|,  |uj|} 

\fz;  w; a,  f]  <  max{ \z] ,  fnJ] }  +Cf 

-  (STq)  If  /  is  tier  0,  let 

J2  =  Cf  '  max{l,  •  max{l,  |z|} 

Ji  =  max{|W|} 

J  ~  J2  Ji 

K2=Cf  max{l, 

Ki  =  max{  fly] } 

K  =K2-\-Ki 

L  =  max{fj]}^2  .  j 
M  =max{f2]}^2 
then  _ 

<  L -h  max{|a|,  (|f| J,  iiT)} 
f/J;  W;  a,  f]  <  M  +  max{  fa] ,  ( ff]  J,  K)} 

It  is  evident  that  all  of  these  bounding  expressions  are  monotone.  For  level 
2,  monotonicity  refers  to  the  partial  ordering  on  functions  induced  pointwise  by 
the  ordering  of  their  level  0  inputs. 

For  the  intuition  behind  (STq)^  consider  that  {\zy  -{-  fiy])  is  approximately 
the  height  of  a  tier  1  value  produced  during  the  course  of  fj,  nested  2-recursions. 
These  values  can  be  used  to  define  a  tier  0  value  using  pi  nested  1-recursions; 
the  resulting  height  is  approximiately  (fz]^  +  +  fa].  On  the  other  hand, 

the  remaining  p2  ~  p  nested  2-recursions  can  be  used  to  repeat  this  process. 
Due  to  the  use  of  tiering,  each  repetition  adds  a  height  of  fz]^  -f-  fw],  giving  a 
total  height  of  fz]^^  ^  +  M*  every  this  is  bounded  by 

+  r«l-  The  expressions  \f\J,K  and  \f]J,K  are  the  length 
and  height  of  tier  0  values  produced  by  applying  f  to  inputs  bounded  by  J  and 
K\  these  tier  0  values  are  treated  in  the  same  way  as  a. 
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Theorem  7.  Every  f  6  2T‘*'  is  nicely  bounded. 

The  proof,  which  is  omitted  here,  is  by  induction  on  the  derivation  of  /. 
Although  the  concepts  used  in  the  proof  are  straightforward,  involving  nothing 
more  complicated  than  monotonicity  of  type  2  functionals,  the  proof  is  quite 
intricate  due  to  the  large  number  of  cases  and  the  nested  expressions  in  the 
definition  of  “nicely  bounded” . 


4  Simulating  ATM  computations 

In  this  section  I  would  like  to  show  that  2r+  is  as  powerful  as  alternating  Turing 
machines  (ATMs).  Ruzzo  proved  that  the  computation  graph  of  an  ATM  is 
essentially  the  same  as  a  uniform  circuit  [16]. 

Computation  graphs  of  ATMs  have  previously  been  used  by  Bloch  to  vali¬ 
date  one  direction  of  a  recursive  characterization  of  NC^  [4].  A  generalization 
to  polylog  depth  was  achieved  by  an  encoding  of  sub  computations  that  gener¬ 
ated  super  poly  nomially  large  output  values;  this  resulted  in  a  characterization 
of  polylog  depth  superpolynomial  size  circuits  [4]. 

Definitions.  The  left  height,  \x\,  of  a  tree  x  is  the  number  of  nodes  in  the 
leftmost  branch  of  x.  Formally,  \0\  =  [1|  =  1  and  ["(a;  *  2/)|  =  1  +  [a;|.  Similarly, 
the  right  height  is  |0]  =  |1]  =  1  and  |(a;  ♦  3/)l  =  1  +  l^/l-  A  path  is  a  tree  that 
is  either  0,  1,  or  is  p  *  0  or  p  *  1  for  some  path  p.  Given  a  vector  of  trees  x,  the 
expansion  oix  is  the  tree  Xx  defined  by:  Xx^  is  xi]  and  is  defined  by 


=  Xa:„. 

Aari . ajfc.l 

= 

Xxi,...,xh,y*z 

=  ^x.,. 

with  each  leaf  replaced  by  Xxi,..,,xh,y 

An  equivalent  definition  is:  Xxi,...,xh+i  is  obtained  by  forming  layers, 

the  top  layer  consisting  of  Xxi,...,xk  and  each  succeeding  layer  being  obtained  by 
replacing  the  leaves  of  the  preceding  layer  with  copies  of  Xxi,...,xh' 

If  we  have  #x  =  k  and  each  a;*  is  a  J9  presentation  (i.e.  a  balanced  tree)  with 
\xi\  =  n,  then  [a;*!  is  approximately  logn,  and  \Xx]  is  approximately  log'^^^n. 
Correspondingly,  \Xx\  is  approximately  2^®®’^“  ”,  a  superpolynomial  size. 

2r+  can  effectively  recurse  over  the  expansion  of  a  by  using  a  nested  recursion 
for  each  xi.  It  is  achieved  by  defining  a  function  E^x;p,v;  f  where  =  k.  The 
superscript  h  indicates  the  operation  that  is  applied  at  each  interior  node  of 
Xx-  The  argument  /  is  an  operation  defining  the  value  at  each  leaf  of  Xx-  The 
argument  p  is  the  path  that  was  followed  to  reach  the  current  node  in  Xx  (i.e. 
to  reach  the  root  of  Xx)-  At  the  root  of  Xx,  the  path  p  may  be  nonempty,  to 
indicate  the  location  of  Xx  in  a  larger  tree,  such  as  in  Xx,z-  When  we  reach  a 
leaf,  for  example,  the  path  p  is  available  for  use  by  the  function  /.  The  argument 
V  is  a  parameter  value  that  is  given  unchanged  as  input  to  /  and  h.  The  value 
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of  B^x;p;  f  is  the  result  of  extending  the  path  p  to  reach  a  leaf  of  applying 
/  at  that  leaf,  and  then  applying  h  at  each  interior  node  of  Xx- 


E^0;p,v]f 

E^l;p,v;f 

Ei{x*yy,p,v;f 


=  /;p,v; 

=  f;p,v] 

=  h;p,  v;  {E^x]p*  0,  v]  /),  {E^r,p*  1,  v;  /) 


0;  ^;  /  =  E^z;  P,v]f 

^]b+if  J 1;  ^;  /  =  E^Z]  p,  v;  f 

*  2/); P,  v;  /  =  Ej^z; p,  v;  (Ap  E^^^z,  x; p,  v;  /) 

The  definition  of  is  analogous  to  the  definition  of  Note  that 

pf  =  max{pf,pf}  and  pf**  =  k. 

Observe  that  the  parameters  v  must  be  in  Ti  according  to  the  type  scheme 
being  used  for  the  development.  If  these  parameters  were  level  1,  then  /  would 
be  level  2  and  E  would  be  level  3. 

The  tree  Xx  can  be  used  as  a  computation  tree  for  a  polylog  time  alternating 
Turing  machine.  To  do  so,  we  must  allow  constant  recursions  to  allow  for  con¬ 
stants  in  the  computation  time  of  the  ATM.  When  K  >  k  below,  we  will  have 
p  >  p  (definitions  are  in  section  3). 


Definition  9.  An  ASPACETIME(P,  k)  machine  is  an  alternating  Turing  machine. 
My  with  the  following  features. 

The  resources  used  by  M  are  determined  by  a  presentation  system  P  and  an 
integer  k.  The  computation  graph  of  M  is  described  by  a  function  x  :  N  (^2)^ 
such  that  K  >  k,  and  such  that  for  exactly  k  value  of  2  €  {1, . . . ,  iiT}  one  has 
Vx  Xi  =  P(x);  and  Xi  is  a  constant  for  all  the  other  values  ofz  6  {!,..., AT}. 
The  computation  tree  on  input  x  is  obtained  from  Xx  by  labeling  the  interior 
nodes  with  either  universal  or  existential  states  according  as  the  depth  of  the 
node  from  the  root  is  even  or  odd,  respectively;  and  by  labeling  each  leaf  with  T, 
F,  L,  or  -iL.  Finally,  M  has  a  constant  number  of  bidirectional  read/ write  tapes. 
The  number  of  cells  used  is  only  bounded  by  the  running  time,  0(\Xx]). 

The  value  returned  by  M  at  a  leaf  labeled  T  is  1;  at  a  leaf  labeled  F  it  is  0; 
at  a  leaf  labeled  L  it  is  (x/2*)  mod  2  where  i  is  the  value  to  the  left  of  the  head 
on  tape  number  1;  and  at  a  leaf  labeled  -iL  it  is  1  ~  ((x/2^)  mod  2)  where  i  is  as 
before.  Each  transition  of  the  machine  is  allowed  to  read,  then  write,  one  bit  on 
each  of  the  tapes. 


Theorem  10.  For  k  >  1,  a  function  is  computable  by  an  0(log*'  n)  depth, 
2^(iog  size  Ue*  -uniform  circuit  iff  it  is  computed  by  an  aspacetime(P,  k) 
machine.  For  k>2,  if  a  function  is  computable  by  an  0{n^~^)  depth, 
size  Ue*  -uniform  circuit  then  it  is  computed  by  an  ASPACETIME(5,  k)  machine. 

Proof  (Sketch).  See  Ruzzo  [16]  for  a  proof  that  uniform  depth  T  size  2"^  is 
equivalent  to  alternating  Turing  machine  time  0{T)  and  space  0(5),  provided 
that  5  =  /2(iogn)'and  T  =  O(logn)  are  suitably  construct  able. 
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The  class  of  ASPACETIME(5,  fc)  machines  is  equivalent  tO'the  class  of  alter¬ 
nating  Turing  machines  using  space  logarithmic  in  |x|  and  at  most  f5(x)'l* 
alternations.  Requiring  the  states  to  alternate  on  each  step,  requiring  exactly 
two  successor  states,  and  requiring  the  input  to  be  accessed  only  at  the  leaves, 
are  well  known  to  increase  the  depth  of  an  i7{log^  7i)-depth  machine  {k  >  1)  by 
at  most  a  constant  factor.  The  size  of  the  computation  tree  can  be  increased  by 
a  constant  amount  by  using  K  >  k  —  additive  constant  factors  can  be  absorbed 
by  multiplicative  constant  factors  because  [x]  >  1.  Although  the  shape  of  the 
computation  tree  is  not  perfectly  balanced  if  |x|  is  not  a  power  of  2,  this  also 
can  be  absorbed  by  increasing  the  depth  by  a  constant  factor:  the  longest  and 
shortest  root-to-leaf  path  lengths  in  B(x)  differ  by  at  most  1,  and  therefore  the 
imbalance  of  the  expanded  tree  is  at  most  a  difference  of  [x]*'  nodes  between 
the  lengths  of  the  shortest  and  the  longest  paths. 

For  the  S  presentation  system,  the  situation  is  analogous.  Although  the  tree 
is  more  unbalanced,  the  shortest  paths  in  the  expansion  are  still  length 
Notice  that  S  presentations  and  the  expansion  Xx  coincide  in  favoring 
the.ieft  leg  of  the  presentation,  n 

Lemma  11.  Using  simultaneous  1-recursion,  one  can  define  a  function  (BIT;  v;  i) 
that  returns  the  {i}th  bit  of  {u},  provided  that  i  is  a  path  and  v  is  a  B  or  S  pre¬ 
sentation  with  {i}  <  |v|.  In  fact,  the  definition  has  =  1. 

The  main  complication  of  a  proof  for  this  lemma  is  that  not  all  root-to-leaf 
paths  of  a  R  presentation  are  the  same  length,  and  therefore  i  does  not  provide  a 
very  direct  way  to  determine  the  bits  of  the  path..  A  detailed  proof  is  omitted. 

Theorem  12.  Let  P  be  either  B  or  S.  For  every  ASPACETIME(P,  A:)  machine 
there  is  an  operation  f  G  ^Tki  that  for  all  x  G  N,  the  output  of  the 
machine  on  input  x  is  given  by  {/P(x)}. 

Proof  The  proof  is  a  direct  application  of  the  function  E. 

Each  possible  machine  state  is  encoded  by  a  constant  in  Tq,  and  each  tape 
state  is  encoded  by  two  paths  a  G  To  and  6  G  To,  such  that  a  contains  the 
bits  at  or  to  the  left  of  the  head  and  b  contains  the  reverse  of  the  bits  to  the 
right  of  the  head.  Thus  the  machine  configuration  consists  of  a  vector  of  tier  0 
paths;  let  0  be  the  initial  machine  configuration.  A  constant  number  of  the  low- 
order  bits  of  these  paths  can  be  manipulated  in  a  suitable  way  using  functions 
such  as  a  ♦  0,  a  ♦  1,  and  L; ;  a.  Specifically  we  can  define  functions  A  and  A 
such  that  ^}; ;  a  is  the  Ith  component  of  the  configuration  obtained  by  choosing 

the  branch  i  G  {0, 1}  from  configuration  a.  Now  using  3^  and  A^  in  the  step 
functions,  simultaneous  1-recursion  can  be  used  to  define  functions  {A'i]p;a) 
giving  the  lih  component  of  the  configuration  that  results  from  configuration  a 
when  path  p  is  followed  in  the  computation  tree.  _  _ 

The  leaf- value  function  /;p,v;  now  can  be  defined  by  obtaining  {A  ;p;0) 
and  testing  the  state  for  T,  F,  L  and  in  the  latter  cases  BIT  is  used  to  obtain 
the  value  from  v.  When  i  is  a  path  represents  the  contents  of  tape  number  1  (up 
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to  the  tape  head)  at  a  leaf  L  or  -»L,  and  v  is  a  copy  of  the  original  input  5(x), 
the  value  of  (bit;  v;  i)  is  the  relevant  bit  of  x. 

The  interior  function  h  is  defined  by  /i;p;a,6  =  “if  (parity  p)  ~  0  then  (a 
AND  b)  else  (a  OR  b)”.  Parity  is  easily  defined  by  1-  recursion,  and  AND  and 
OR  are  defined  using  the  conditional. 

Lei  K  >k  and  x  be  as  in  the  definition  of  an  ASPACETIMe(P,  k)  machine:  x 
is  a  vector  of  K  —  k  constants  and  k  copies  of  R(x)  such  that  the  expansion  of 
X  is  the  computation  graph  of  M  on  input  x.  Letting  i  satisfy  Xi  =  R(x),  the 
ASPACETIME  output  is  E^X]p,Xi]f. 

The  p2  for  the  defined  function  is  which  is  K.  Its  1-rank  is  the  1-rank 
of  / .  Since  /  is  a  imposition  of  the  initial  functions  with  BIT  and  A* ,  one  has 

Considering  that  all  but  k  of  the  K  recursions  defining  Ek  are  substituted 
by  the  constant  values  among  x,  one  has  that  p2  for  the  defined  function  is  at 
most  k.  0 

This  establishes  one  direction  of  the  main  theorems. 

Theorem  13.  Uniform  polylog  parallel  time  is  contained  in  the  level  1  section 
of  B/2T'^.  In  fact,  for  all  k>  I,  uniform  parallel  time  0(log*  n)  is  contained  in 
the  level  1  section  of  Bj2T^^. 

Theorem  14.  Uniform  polynomial  parallel  time  is  contained  in  the  level  1  sec¬ 
tion  of  S/2T'^.  In  fact,  for  all  k  >2,  uniform  parallel  time  0(n*)  is  contained 
in  the  level  1  section  of 

Proof.  These  two  corollaries  are  a  direct  consequence  of  the  preceding  simu¬ 
lation  theorem  together  with  the  modified  form  of  Ruzzo’s  theorem.  Notice  that 
the  preceding  theorem  does  not  require  any  bound  on  the  space  used  by  the 
ATM.  □ 

5  Simulating  2T+  operations 

The  second  direction  to  be  proved  is  that  2r+  operations  can  be  computed  by 
ATMs  within  suitable  resource  bounds. 

Theorem  15.  The  level  1  section  of  B/2T'^  is  contained  in  uniform  polylog 
parallel  time.  In  fact,  for  k  >  1,  the  level  1  section  of  P/2Tj^j  is  contained  in 
uniform  parallel  time  n). 

Theorem  16.  The  level  1  section  of  S/2T^  is  contained  in  uniform  polynomial 
parallel  time. 

The  use  of  circuits  for  a  formal  proof  of  these  theorems  would  be  awkward 
because  the  uniformity  conditions  would  require  a  mapping  from  the  implicitly 
uniform  structure  of  2T+  derivations  to  an  explicit  machine  computing,  say,  the 
extended  connection  language  of  the  circuit  family.  However,  in  other  respects 
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circuits  provide  a  direct  way  to  understand  B/2T'^  operations.  Therefore,  let  me 
use  circuits  to  give  an  informal  idea  of  the  result. 

Consider  circuits  with  many  output  bits  and  with  oracle  gates  for  the  level 
1  inputs.  Each  oracle  gate  is  labeled  with  the  name  of  one  of  the  level  1  inputs, 
and  has  any  number  of  bits  input  to  it  and  a  corresponding  number  of  bits 
output  from  it.  Furthermore,  we  maintain  the  invariant  that  each  rootrto-leaf 
path  passes  through  at  most  one  oracle  gate.  This  invariant  corresponds  to  the 
fact  that  the  types  of  level  1  inputs  are  predicative:  the  result  of  applying  a  level 
1  input  cannot  be  used,  either  directly  or  indirectly,  as  the  input  to  another  level 
1  oracle  application.  An  exception  occurs  when  C  is  used,  as  in  (h;  u,  v,  W]  /,  g)  = 
(/;  (C;  w,  V,  w\  (^u)); )  —  here  the  To  output  of  gu  is  being  used  indirectly  as  an 
input  to  /.  Such  a  usage,  however,  can  always  be  rewritten  as  in  (h;  u,  v,  w\  /,  g)  = 
C;  {fu)y  ifv),  ifw);  (gu).  Intuitively,  the  presence  of  at  most  one  oracle  gate  along 
any  given  path  allows  us  to  unroll  2-  recursions  as  iterations.  For  unrolling  1- 
recursions  one  similarly  requires  that  each  literal  testing  a  tier  0  input  cannot  be 
used  indirectly  as  an  input  to  an  oracle  gate.  For  example,  level  0  oracles  testing 
input  bits  are  not  descendants  of  level  1  oracle  gates. 

To  show  that  there  is  at  most  one  oracle  gate  along  each  root  to  leaf  path, 
one  assigns  a  tier  to  each  gate  of  the  circuit:  the  tier  of  a  leaf  is  the  tier  of  the 
input  being  tested  at  that  leaf,  or  is  1  if  the  leaf  is  a  constant;  the  tier  of  an 
AND  or  OR  gate  is  the  minimum  of  the  tiers  of  its  inputs;  and  the  tier  of  a  level 
1  oracle  is  0  (since  the  oracle  function  has  admissible  type  with  output  in  To). 
The  invariant  is  that  the  gates  input  to  the  level  1  oracles  are  all  tier  1;  this 
corresponds  to  the  fact  that  admissible  level  1  oracles  have  all  tier  1  inputs. 

In  the  actual  circuit  corresponding  to  a  2T*^  operation,  the  number  of  inputs 
to  the  oracle  gates  is  bounded  by  the  size  of  the  circuit,  and  this  is  bounded 
using  the  Bounding  Lemma.  The  size  of  the  circuit  is  not,  naively,  2^  where  I  is 
the  length  of  computed  values,  because  many  of  the  bits  in  a  computed  value 
are  simply  copies  of  the  bits  of  the  input. 

It  is  easy  to  construct  circuits  for  the  initial  functions.  For  1-recursion,  one 
replaces  the  leaves  of  the  circuit  for  h*  with  copies  of  the  circuit  for  h* .  The  sub¬ 
stituted  leaves  are  oracles  for  the  level  0  input  bits.  The  substitution  is  repeated 
[u]  —  1  times,  where  v  is  the  input  being  recursed  on.  Finally,  one  replaces  the 
resulting  leaves  with  a  circuit  that  uses  the  input  bit  at  that  leaf  to  select  the 
output  from  either  a  copy  of  or  a  copy  of  h^.  The  resulting  circuit  depth  is 
[i;]  times  what  it  used  to  be;  for  the  B  presentaton  [v]  =  O(logn).  Although 
at  first  it  might  appears  that  this  construction  could  generate  I’v]  oracle  gates 
along  a  root-to-leaf  path,  in  fact  this  does  not  happen  because  the  leaves  being 
replaced  correspond  to  tier  0  inputs,  and  these  will  not  be  the  descendants  of 
oracle  gates. 

For  2-recursion,  start  with  a  circuit  for  h*:  it  is  parameterized  by  the  bits 
for  the  recursion  variable  x,  it  hats  height  log*'”^  n,  and  it  contains  oracle  gates 
for  accessing  the  level  1  inputs.  To  get  a  circuit  corresponding  to  the  recursion, 
replace  each  oracle  gate  for  the  critical  terms  with  a  copy  of  the  circuit  for  /i*, 
but  parameterized  by  either  Lx  or  Rx.  This  substitution  is  repeated  [x]  -  1 
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times.  At  the  end,  there  is  only  one  parameter  bit  of  x  given  as  input  to  the 
oracle  gate.  (Of  course  the  oracle  gate  can  have  other  input  bits  corresponding 
to  non-recursion  inputs).  Then,  replace  these  oracle  gates  with  a  circuit  that  uses 
the  parameter  bit  x  to  select  the  output  from  either  a  copy  of  or  a  copy  of  . 
By  placing  the  copies  of  and  in  parallel,  one  preserves  the  invariant  that 
each  root-to-leaf  path  contains  at  most  one  oracle  gate.  It  is  easy  to  see  that  this 
invariant  is  also  maintained  by  the  repeated  substitution  of  h*.  Because  there 
was  originally  at  most  one  oracle  gate  along  each  path,  the  height  of  the  resulting 
circuit  is  at  most  ([a:]  -  !)•  [h*]  +max{|‘/i°],  The  size  of  the  circuit  does 
increase  rapidly  up  to  due  to  the  presence  of  multiple  oracle  gates  for 

the  critical  terms. 

In  the  case  of  S/2T^,  one  can  follow  exactly  the  same  proof,  again  obtaining 
bounds  on  the  height  of  the  circuit  in  terms  of  \x].  The  condition  that  there  is 
at  most  one  oracle  gate  along  each  root-to-leaf  path  is  now  considerably  more 
severe  relative  to  the  size  of  the  circuit:  a  typical  root-to-leaf  path  now  passes 
through  a  much  larger  number  of  nodes. 

6  Conclusion 

The  results  of  this  paper  confirm  a  direct  relationship  between  type  2  recursion 
and  parallel  computation.  It  has  been  shown  that  in  defining  a  recursion  class, 
the  output  length  can  be  treated  separately  from  the  complexity  of  the  bits 
in  the  output.  This  suggests  that  output  length  is  not  a  barrier  to  obtaining 
natural  recursive  characterizations  of  parallel  complexity  classes.  Furthermore, 
the  results  provide  additional  evidence  that  the  power  of  type  2  recursion  can 
be  sharply  curtailed  by  appropriately  ramified  structures. 

The  reduction  in  output  length  was  achieved  without  reducing  the  space 
available  to  the  corresponding  ATM  or  circuit  computation.  If  the  ATM  space 
were  O(logn),  a  characterization  of  NC  would  be  obtained.  Thus  an  evident 
challenge  for  future  research  is  to  show  how  to  characterize  lower  complexity 
classes,  such  as  NC,  in  a  similar  way. 

The  definition  of  2T^  has  a  coincidence  between  the  tier  of  the  recursion 
variable  and  the  type  level  of  the  recursion  allowed  on  this  variable.  At  the  same 
time,  Leivant  [10]  has  shown  that  two  tiers  are  as  good  as  infinitely  many  for 
characterizing  poly  time  with  type  1  recursions;  and  Leivant  and  Marion  [12] 
have  shown  that  three  tiers  are  as  good  as  infinitely  many  for  characterizing 
polyspace  with  type  2  recursions.  This  makes  one  wonder  whether  the  use  of 
higher  tiers  has  some  intrinsic  relationship  to  the  use  of  higher  types. 

The  formal  use  of  presentations  such  as  B  and  S  seems,  to  this  author,  con¬ 
nected  to  the  question  of  domain  ordering  from  finite  model  complexity  theory 
[8].  Intuitively,  any  recursively  defined  presentation  system  has  implicit  in  it  an 
ordering,  i.e.  the  ordering  given  by  the  recursive  structure  of  the  presentations. 
Thus,  the  thesis  that  various  complexity  classes  can  be  naturally  obtained  by  con¬ 
sidering  different  presentations  for  a  single  class  of  operations,  seems  related  to 
the  thesis  that  various  ordering  relations  will  induce  different  complexity  classes 


267 


under  a  fixed  iterated  first-order  language.  We  cannot  rule  out  the  possibility 
that  a  single,  natural,  recursion  class  C  could  be  defined  such  that  many,  if  not 
all,  natural  complexity  classes  are  induced  by  various  presentation  systems  for 
(7. 
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Abstract.  In  this  paper  we  examine  type  2  analogs  of  the  type  1  polynomial  hierarchy  and 
show  some  limitations  on  finding  a  completely  faithful  type  2  analog.  We  survey  most  of  the 
notions  of  type  2  poly-hierarchies  already  proposed  in  the  literature  and  present  two  natural 
definitions  of  type  2  poly-hierarchies.  We  also  introduce  various  resource  bounded  reductions 
between  functionals  of  type  2. 


1  Introduction 


Functionals  of  type  2  are  total  functions  which  take  type  1  functions  and  natural  numbers  as  arguments 
and  output  a  natural  number.  In  recent  years  some  generalizations  of  type  1  poly-time  to  higher  types 
have  been  studied.  Many  important  classes  other  than  poly-time  are  also  studied  in  type  1  complexity 
theory.  One  example  is  polynomial  hierarchy,  [13].  In  this  paper  we  investigate  possible  generalizations 
of  type  1  polynomial  hierarchy  to  type  2. 

Some  possible  generalizations  are  defined  and  studied  in  [14].  However  there  are  several  definitions 
of  type  2  polynomial  hierarchies  in  the  literature  and  it  is  not  clear  which  is  the  right  one.  Many 
questions  about  the  properties  and  interrelationships  of  these  proposed  hierarchies  are  open,  see  [14]. 
In  this  paper  we  try  to  investigate  the  issue  systematically  by  writing  down  some  natural  conditions  for 
type  2  polynomial  hierarchy,  in  the  same  way  as  conditions  8. 1-8.3  in  [8]  are  proposed  for  any  class  of 
feasible  functionals.  Surprisingly  we  find  that  these  natural  conditions  lead  to  contradiction.  Therefore 
there  can  be  no  type  2  polynomial  hierarchy  which  possesses  all  these  properties,  nevertheless  some 
attractive  generalizations  are  possible.  We  survey  the  previous  definitions  in  this  light  and  present  some 
new  results  about  them.  We  later  present  two  possible  definitions  of  type  2  polynomial  hierarchies,  each 
of  which  is  natural  from  a  different  viewpoint.  These  definitions  have  appeared  in  literature  in  slightly 
variant  forms.  These  notions  also  arise  naturally  in  the  context  of  type  2  feasible  functionals  definable 
in  various  systems  of  bounded  arithmetic  which  we  study  elsewhere. 

The  rest  of  this  paper  is  organised  as  follows.  In  section  2  we  show  limitations  on  finding  a  faithful 
analog  of  type-1  poly-hierarchy  in  type  2.  In  section  3  we  review  various  definitions  proposed  in  the 
literature  for  type  2  poly-hierarchies.  In  the  next  section  we  prove  some  results,  about  these  proposed 
classes  left  open  in  the  literature.  In  section  5  we  extend  the  various  notions  of  time  bounded  reducibil- 
ities  between  type-1  functions  to  analogous  reductions  between  type  2  functionals.  These  notions  of 
reductions  are  shown  to  be  different  on  the  space  of  computable  functionals.  In  section  6  we  examine 
in  some  detail  a  notion  of  type  2  poly-hierarchy  which  is  used  by  V.  Harnik  in  [9]  to  generalize  some 
results  of  [5].  In  section  7  we  investigate  another  natural  notion  of  type  2  poly-hierarchy  and  prove 
some  of  its  properties.  The  next  section  relates  this  poly-hierarchy  with  a  class  already  considered  in 
the  literature.  Finally,  section  9  contains  some  concluding  remarks.  For  the  terminology  and  notation 
not  defined  in  this  paper  we  refer  the  reader  to  [10]. 


2  Desirable  conditions  and  limitations  for  type  2  polynomial  hierarchy 


In  this  section  we  propose  some  desirable  conditions  for  a  type  2  polynomial  hierarchy,  in  the  same 
spirit  as  the  necessary  conditions  proposed  in  [8]  for  a  class  of  feasible  functionals.  These  conditions  are 
a  straightforward  lifting  of  some  characteristic  properties  of  the  type  1  polynomial  hierarchy. 
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^2,k’  ^2,k  denote  the  classes  of  type  2  predicates  of  level  of  the  proposed  polynomial  hierarchy 

(  ^Ik  consists  of  complement  of  predicates  in  and  let  Oj  be  the  class  of  type  2  functionals  of 
level  of  the  proposed  polynomial  hierarchy.  rj.iTjf  denote  the  predicates  in  the  A:*''  level  of  the  type  1 
polynomial  hierarchy.  Dj  denotes  functions  of  k^^  level  of  this  hierarchy.  More  specifically,  X'J,  /7f  denote 
polytime  computable  predicates  and  Dj  denotes  poly-time  functions,  Dj  denotes  functions  computable 
in  polytime  using  Tj  oracle.  (This  definition  of  Dj  is  slightly  different  from  the  one  given  in  [2]). 

The  following  are  expected  conditions  for  a  type  2  polynomial  hierarchy. 

0,1  valued  functionals  in  The  class  Df  o  satisfies  the  conditions  laid  down  for  a 
class  of  feasible  functionals. 

2-  °2,jb  =  °2,o(-^2,fe)  (=  ^  closure  of  and 

FVom  this  it  follows  that  the  class  Dj  ^  is  closed  under  A  abstraction  and  application. 

3.  All  type  1  functions  constructed  from  the  functionals  in  type  1  functions  in  Oj  and  A  calculus 
operations  are  in  Dj. 

4.  Each  contains  0-1  valued  functionals  in  0^  ^,  and  each  is  closed  under  Djo  functional 

bounded  existential  quantifier  in  natural  number  Wable,  that  is’  a  quantifier  of  the  type  3|a:J  <  |<^| 
where  x  is  a  natural  number  variable  and  ^  is  a  functional  in  Oj  q.  Similarly  each  iJf  ^  is  closed 
under  the  corresponding  universal  quantifier.  ’  ' 

Theorem  1  Existence  of  a  type  2  polynomial  hierarchy  with  properties  (l)-(4)  above  implies  the  collapse 
of  type  1  polynomial  hierarchy. 

Proof;  Let  A{z)  ^  3|xj  <  p(|2:|)V|y|  <  p(niux{|2:|,  |x|})[B(x,y,  2:)]  ,  where  p  is  a  polynomial  and 
a  poly-time  predicate,  be  a  complete  predicate. 

Consider  the  type  2  predicate  F(/,z)  defined  as  Sjx]  <  p{\z\)[f{z,x)  ~  Ij 

F  is  in  1  level  of  any  type  2  polynomial  hierarchy  that  satisfies  conditions  (l)-(4)  laid  down 
above.  (By  (4),  as  predicate  P{f,z,x)  =  {f{z,x)  =  1]  e  DIq  ) 

Let  £{z,x)  be  the  characteristic  function  of  predicate  V|y(  <  p{max{\z\,  |x|})[jB(x,  y,  z)]. 

Therefore  /  G  Of . 

Now  the  type  1  predicate 

=  31x1  <p(|z[)[/(z,x)  =  ll 
=  3ix|  <  p{\zmy\  <  P{max{\zl  \x\})[B{x,  y,  z)]] 

=  Aiz) 

Since  A{z)  can  be  constructed  using  i7f  j  functional,  Of  function  and  substitution  and  since  Tf , 
preserves  Of,  we  have  that  Tf  C  Of.  This  implies  Tf  =  Tf .  □ 

In  view  of  this  theorem  we  have  to  give  up  the  search  for  a  type  2  polynomial  hierarchy  with  all  the 
above  properties.  However  the  notion  of  type  2  polynomial  hierarchy  has  still  been  useful  (see  [9]),  so 
we  shall  consider  the  hierarchies  that  satisfy  only  a  reasonable  subset  of  the  above  conditions. 


3  Previous  approaches 

^  this  section  we  recall  various  notions  of  type  2  polynomial  hierarchies  that  have  been  proposed  so  far 
in  the  literature.  As  shown  in  the  previous  section  each  of  these  hierarchies  must  fail  to  satisfy  at  least 
one  property  from  (l)-(4).  In  the  next  section  we  study  the  properties  of  these  hierarchies  in  detail  and 
will  answer  several  open  question  about  them.  We  recall  the  following  definitions  from  [14]. 

Type  2  relations  are  0,1  valued  functionals  of  type  2.  Class  BFF  [5]  is  referred  to  as  class  POLY  in 
[14],  we  shall  therefore  use  the  two  terms  interchangeably  in  this  paper. 

Definition  1  The  class  of  polynomially  bounded  relations  is  defined  as  the  least  class  of  predicates 
which  contains  0-1  valued  basic  feasible  functionals  and  is  closed  under  length  bounded  quantifications 
of  the  form  3|xi  <  li^(f,y)l  and  V|x|  <  i^(f,y)|  ,  where  <f>  €  BFF. 
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Townsend  [14],  classified  polynomially  bounded  relations  in  a  hierarchy  as  follows.  In  order  to  keep 
the  notation  distinct  we  shall  describe  levels  of  this  hierarchy  using  boldface. 

Definition  2  is  the  class  of  relations  in  BFF. 

-^n+l  ”  of  relations  definable  hyprefixing  a  31a;|  <  10(f,y)i  quantifier  to  relations  in  ijJ, 

is  the  class  of  relations  definable  by  prefixing  a  V|x|  <  |<^(f,y)|  quantifier  to  relations  in  rg. 
(here  <f>  €  BFF) 


It  is  left  open  in  [14]  to  answer  if  the  above  hierarchy  is  proper.  It  is  also  conjectured  in  [14]  that 
B  levels  of  the  above  hierarchy  are  not  closed  under  length  bounded  existential  quantification.  We  will 
answer  both  these  questions  in  the  next  section. 


Definition  3  The  relativized  class  n  >  0  *5  defined  as  follows.  =  {F(f,y)|  there  is  some 
G  €  and  F(f,y)  =  (j(f,p,y)}.  iJP  is  defined  similarly. 

Victor  Harnik  has  defined  a  similar  hierarchy  by  taking  g  to  be  type  1  poly-  hierarchy  functions  of 
appropriate  level  and  G  €  BF F  in  the  above  definition.  We  shall  study  that  hierarchy  in  section  6. 

Ih  [14],  yet  another  hierarchy  is  defined  as  follows. 

Definition  4  Fg  =  B^’^. 

iZn  =  U,  n>0 


In  the  next  section  we  will  show  that  this  hierarchy  collapses  to  the  first  level.  This  answers  another 
question  of  [14]. 


4  New  results  on  the  previous  work 


In  this  section  we  investigate  the  properties  of  the  hierarchies  defined  in  [14].  These  results  will  also  be 
useful  in  the  later  sections  of  this  paper. 


4.1  Separation  of  type  2  classes 

The  following  proof  shows  the  separation  among  levels  of  the  hierarchy  of  polynomially  bounded  rela¬ 
tions,  however  the  same  idea  can  be  used  to  show  separation  of  any  two  classes  for  which  analogous 
type  1  classes  are  different  under  some  oracle.  The  idea  here  is  to  regard  the  oracle  as  type  1  input  to 
a  functional.  It  has  been  observed  independently  in  the  literature  that  oracle  separation  can  be  used 
to  show  separation  for  type-2  classes  though  we  have  not  seen  a  detailed  argument.  We  present  full 
details  here  so  that  it  becomes  clear  exactly  when  oracle  separation  can  be  converted  into  an  actual 
type  2  separation.  To  use  it  in  the  later  sections  we  just  notice  that  all  conditions  for  this  argument  to 
be  applicable  are  fulfilled. 

In  keeping  with  the  terminology  of  [14]  we  use  lightface  symbols  to  denote  type  1  complexity  classes 
and  boldface  symbols  to  denote  analogous  type  2  classes. 

Lemma  1  ;  rg  C 

(C  stands  for  a  proper  subset.  ) 

Proof;  Note  that,  B^  denotes  the  levels  of  the  type  1  polynomial  hierarchy  and  rg  denotes  the  levels 
of  the  hierarchy  of  polynomially  bounded  relations. 

Let  be  a  0,1  function  such  that  B^^^f  C  Xjfi,  (from  [15],  such  a  function  exists). 

Let  f(x)  be  a  0,1  function  €  B^f^  -  B^’^. 
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By  the  characterization  of  polynomial  hierarchy  we  have  that 

[fix)  =  1]  =  Pll/n+ii  <  N*V|y„[  <  |x|^..QbiI  <  |a:|*/i^'(x,2/i,...yn+i)]. 

<5  is  3  if  n  is  odd,  otherwise  Q  is  V.  i?  is  a  PTIME  predicate  computable  in  0(m*)  time  relative  to 
oracle  g. 

Define  functional  G(a;,  j/i , . . . ,  i/n+i ,  h)  as  follows.  The  machine  computing  G  on  input  x,  2/1 , . . .  2/„+i ,  h 
simulates  R  on  x,yi,..  .2/n.fi  with  oracle  h.  If  an  answer  >  1  is  returned  by  oracle  or  the  computation 
time  exceeds  \max{x^yi,.  -  then  the  machine  stops.  So  G  is  in  POLY. 

Let  Fih,  x)  =  3\yn+i  \  <  |x!*V|yn!  <  \x\^  •  •  •  Qlvil  <  W*G(x,  .  yn+i,  h) 

Q  is  3  if  n  is  odd,  otherwise  Q  is  V.  Observe  that  F{g,  x)  =  /(x). 

F  6  We  claim  that  F  i  rg. 

To  see  this  assume  towards  a  contradiction  that  F  6 

Now  Fih,x)  ~3\yn+i\  <  ls„+i{x,/i)|V|y„|  <  |an(a:,J/n+i,  A)l  •  •  • 

. .  .Qbit  <  j5i(x,y„+i,2/n, .  • .  ,2/2,  h)\Tix,yi,. . .  ,yn+i,  ^), 
where  each  si,S2,. .  .Sn+i,T  are  computable  by  0(7n*')  POTMs  (In  fact  they  are  in  POLY). 

(On  0,1  valued  functions  k,  |si(x,  2/„+i, . . . ,  2/i+i,  ^)l  is  bounded  by  a  polynomial  of  degree  k  in  length 
of  type  0  input,  and  each  Sj  is  computable  in  polynomial  time  with  oracle  h). 

Therefore,  by  the  characterization  theorem  of  relativized  polynomial  hierarchy,  we  have 

[F(h,x)  =  1]  =  [3|ji„+.|  <  W‘V||,„|  <  |xl‘"  ...Qlj/il  <  |x|‘""'r(x.yi . y„+i.A)] 

OR 

[^(S.x)  =  1]  =  13|y„+i|  <  |x|‘V|y„|  <  |x|‘’ .. .C}|y,|  <  |x|‘“-"'ri»(x,y, . y„+i)l, 

where  Tf  is  a  polynomial  time  predicate  with  oracle  g. 

Since  /(x)  =  F(p,x)  this  implies  that  /(x)  6  ,  a  contradiction.  □ 

In  [14]  classes  NP,  Co-NP,  PH,  PSPACE  are  also  defined.  We  do  not  give  their  definition  here 
but  make  the  following  observation. 


Corollary  1  NP  #  Co  -  NP,  ijg  U  i:g  C  n  ,  PH  C  PSPACE. 

Proof:  There  are  oracles  showing  relativized  separation  among  corresponding  type  1  classes.  Also  the 
definition  of  these  type  2  classes  is  such  that  when  all  the  type  1  inputs  are  fixed  to  some  set  A  (a  0  - 1 
valued  function)  then  the  resulting  classes  are  the  corresponding  type  1  classes  relativized  to  oracle  A. 
□ 

Remark:  The  proof  above  does  not  show  separation  between  and  Indeed,  the  theorem 

below  shows  that  this  hierarchy  collapses  to  POLY. 


4.2  Collapse  of  27^  hierarchy 

Definition  5  We  define  k  count  tape  space  bounded  machines  in  the  same  way  as  the  machines  to 
compute  functionals  of  class  Ci  defined  in  [10]  except  that  only  their  workspace  (not  runtime)  needs  to 
be  bounded  by  a  polynomial  in  length  of  the  contents  of  k*^  count  tape.  The  space  bound  also  applies  to 
output  tape  and  to  oracle  query  tapes  but  not  to  the  oracle  answer  tapes.  We  define  class  PSPACE' 
to  be  the  class  of  all  type  2  relations  whose  characteristic  functional  can  he  computed  by  a  k  count  tape 
0(mP)  space  hounded  machines  for  some  k  and  p. 

Let  PSPACE' [y]  be  the  class  PSPACE'  relativized  to  type  1  function  g  and 
PSPACE'  =  Ug  PSPACE' [5]. 

It  is  easy  to  see  that  for  all  n,  i7g[ff]  C  P SPACE' [j]  and  therefore  Lg  C  PSPACE'. 

Theorem  2  PSPACE'  =  POLY. 

Proof:  Let  F(/,n)  €  PSPACE'.  There  exists  a  g  such  that  F(/,n)  =  G(/,5,n)  and  G(/i,/2,x)  is 
computable  by  a  machine  k  count  tape  0(771**)  space  bounded  machine  M. 
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Observe  that  if  the  space  used  by  some  machine  is  bounded  by  Z,  then  there  is  an  equivalent  machine 
whose  runtime  can  be  bounded  by  c^,  for  some  constant  c  which  depends  on  the  machine  only.  Suppose 
that  Li  is  the  length  of  the  largest  answer  returned  by  machine  M  computing  (7,  on  input  /,  g,  n.  Let 
L  ~  then  space  used  by  M  is  bounded  by  hence  time  used  by  it  is  bounded  by  c^*’. 

Consider  a  function  g'  which  on  input  of  length  x  returns  the  output  of  length  c®*’,  that  is  Iff'(n)|  = 

Let  g"  be  a  function  such  that,  ^"(2n)  =  g{n)  and  g"(2n  +  1)  =  g‘{n). 

Our  objective  now  is  to  design  a  time  bounded  count  tape  machine  which  on  input  /,  g'\  n  can 
simulate  M  on  input  ftg,n.  This  is  possible  as  the  function  g  is  coded  in  the  function  g".  Further  the 
machine  to  be  designed  can  also  query  g"  on  appropriate  (odd  numbered)  inputs  to  receive  large  enough 
answer  so  that  its  runtime  (not  just  the  space  used)  is  also  bounded  by  a  polynomial  in  the  length  of 
the  largest  count  tape  contents. 

Consider  a  A:  + 1  tape  O(m’)  machine  M\  {q  >  p^'it>  q[t^  >  2t  + 1]),  computing  G'(/,  h,  n),  which 
on  input  f,h,n  first  queries  h  on  2.maa;{n,p}  +  1  and  starts  simulating  M  on  input  Whenever 

M  queries  g  at  u,  then  M'  queries  h  at  2u  and  at  2.h{2u)  + 1  and  returns  h{2u)  to  Af  as  an  answer.  If 
M  queries  /  at  u  during  the  simulation  then  M*  queries  /  at  v  and  h  at  2/(v)  +  1  and  returns  f{v)  as 
the  answer.  Further  whenever  M'  queries  h  at  any  odd  value  2a:  + 1  then  M'  halts  if  |/i(2rc+ 1)|  < 

It  is  easy  to  see  that  on  any  input  /,  h,  n  the  runtime  of  Af '  at  all  instants  during  the  computation 
is  bounded  by  a  polynomial  in  |max{n, yi, . . . ,  yi}|,  where  i/i  are  all  the  oracle  answers  returned 

till  that  instant.  Notice  that,  since  Af  is  a  fc  count  tape  machine  therefore  all  queries  of  Af'  lie  in  block 
[k,  k  +  1]  or  lower. 

Therefore  by  corollary  1,  [10]  we  have  that  G'{f,h,n)  is  in  POLY.  Also  F{f,n)  =  G'{f,g",n). 
Hence  F(/,n)  ePOLYj^"].  □ 

Corollary  2  For  oH  ti  >  0,  ^  =  POLY. 

It  is  of  independent  interest  to  note  the  following. 

Observation:  A  Rinctional  F{f,  x)  e  PSPACE'  iff  it  is  computable  by  QTMs  in  space  bounded  by 
P(|/|,  jxj),  for  some  second  order  polynomial  P. 

This  is  shown  easily  because  a  functional  A(/,x),  such  that  \A{f,x)\  =  P(|/|,  |x|),  is  in  PSPACE'. 


4.3  Closure  under  length  bounded  quantification 

It  is  conjectured  in  [14]  that  the  levels  of  the  class  of  polynomially  bounded  relations  are  not  closed 
under  length  bounded  existential  quantification.  We  prove  a  more  general  result  below  which  implies 
the  conjecture. 

Lemma  2  There  is  a  functional  obtained  by  prefixing  k  +  1  POLY  bounded  existential  quan¬ 

tifiers  to  a  POLY  predicate,  which  is  not  representable  by  any  POTM  computable  predicate  quantified 
k  times  (by  any  combination  of  3  and'i  quantifiers)  using  POTM  functionals  bounded  quantifiers. 

Proof:  Let  A{f,z)  ~  3|xi|  <  \z\3\x2\  <  |/(xi)| • . <  |/(xit)([/(xfc+i)  is  even  ]...(!) 

For  each  formula 

Qi\y\\  <  ki(^,/)|-..Qjbli/fc|  <  k(yi,...,3/jb-i,^,/)|P(yi,...,2/)t,z,/)  ...(2) 

where  si,. .  .,sjfc,  JR  are  computable  by  POTMs,  and  each  Qi  6  {3,V},  we  construct  an  /  such  that 
for  some  large  enough  z,  formula  A(/,  z)  is  true  iff  (2)  is  false. 

Let  t  be  such  that  si, . . . ,  sjk,  P  are  computable  by  0(m*)  PQTMs.  We  describe  the  construction  of 
/  in  the  following. 

In  the  first  step,  we  proceed  as  follows.  Choose  a  z  such  that  \z\*  <  2l*L  Run  a  POTM  computing 
81  on  input  /,  z  answering  all  queries  to  /  as  1.  Define  /  to  be  1  on  all  the  inputs  queried  during  the 
computation.  Define  /  to  be  an  odd  number  Afi  at  all  unqueried  strings  of  length  <  \z\.  Mi  is  chosen 
such  that  it  satisfies  the  property  2t^»l  >  |Afi|*  x  2l^l*.  Let  vi  be  one  string  such  that  |uij  <  \z\  and 
/(vi)  =  Afi  .  (vi  exists  by  the  choice  of  z). 

In  the  second  step  we  proceed  as  follows.  For  all  yi,lyi|  <  \z\^  run  a  POTM  computing  S2{yi,f,z) 
and  answer  all  queries  to  f  at  inputs  where  /  has  not  been  defined  so  far,  as  1.  /  is  defined  to  be 
1  at  the  queried  string,  after  answering  such  a  query.  After  the  computation  of  sj  define  /  at  ail  the 
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strings  of  length  <  |Mi|,  on  which  /  is  not  defined,  as  M2.  Mg  is  chosen  to  be  an  odd  number  such 
that  >  IM2I*  X  Let  V2  be  a  string  such  that  IU2I  <  iMij,  and  /(V2)  =  M2,  such  a  V2  exists 

by  choice  of  Mi  at  the  previous  step. 

In  the  step  we  do  the  following;  for  all  sequences  such  that  |yi|  <  IMi_2iS-.. 

...,|yi_i|  <  |Mi_2|*,  run  a  POTM  computing  During  this  computation  if  /  is 

queried  at  an  input  on  which  it  has  not  been  defined  then  1  is  returned  as  the  answer  to  the  query  and 
f  is  defined  at  the  queried  string  to  be  1.  At  the  end  of  this  simulation,  we  choose  an  odd  number  M,- 
for  which  2l^*l  >  x  |Mi|*  x  2l^‘-'l‘  holds.  Define  /  to  be  at  all  the  inputs  of  length  <  !Mi_i-j,  where 
it  has  not  been  defined  so  far,  as  Mj.  Let  Vi  be  one  string  such  that,  |ui|  <  iMi_i|  and  /(u<)  =  M,-.  (by 
the  choice  of  Mi_i  at  the  previous  step  such  a  string  exists,  as  total  number  of  strings  queried  in  the 
simulation  upto  and  including  step  is  <  x  |M<_i|^  x  ) 

After  k  steps  we  obtain  M*. 

Notice  that  by  the  preceding  construction, '  the  truth  of  (2)  on  input  z,f  depends  only  on 
■R(yi,...,yfc,2,/)  such  that  |yi|  <  lMjfe_il*, . . . , <  |Mfc_ii*.  Determine  the  truth  of  formula  (2) 
by  running  a  POTM  computing  i2(yi, . . . ,  y*,  2:,  /)  on  all  yjs  such  that  |yi|  <  |Mfc_i|*  and  answering  all 
queries  to  /  where  it  is  not  defined  as  1.  /  is  defined  to  be  1  at  such  a  queried  input  after  answering  the 
query.  Observe  that  M*  is  chosen  to  be  large  enough  so  that  there  is  a  string  of  length  <  |Mfc|  which 
is  not  queried  by  R  on  any  input  and  is  also  not  queried  in  any  of  the  previous  k  steps  and  /  is  not 
defined  on  this  string. 

Let  u  be  a  string  of  length  <  |Mjfc|,  not  queried  in  any  of  the  previous  steps  and  on  which  /  is  not 
defined.  Define  f(u)  as  4  (some  even  value)  iff  the  formula  (2)  is  false.  Define  the  value  of  /  to  be  I'at 
all  the  inputs  on  which  it  has  not  been  defined  so  far.  (By  choice  of  u,  truth  of  (2)  is  independent  of 
/(li)  )• 

If  the  formula  (2)  is  true  then  A{f,  z)  is  false  as  f{x)  is  odd  for  all  x,  on  the  other  hand  if  the  formula 
(2)  is  false  then  (1)  is  true  via  xi  =  Vi , X2  =  V2,  •  •  •  a;*  =  v*,  =  u.  □ 

Corollary  3  Levels  of  the  hierarchy  of  polynomially  bounded  relations  are  not  closed  under  POLY 
length  bounded  existential  quantification  of  natural  number  variables.  In  fact,  for  each  level,  there  is  an 
infinite  hierarchy  based  on  the  number  o/POLY  bounded  existential  quantifiers  that  can  prefix  formulas 
of  that  level. 

5  Feasible  reducibility  between  functionals 

In  order  to  define  the  polynomial  hierarchy  for  type  2  functionals  we  need  to  define  some  notion  of 
resource  bounded  reduction  between  functionals.  Below  we  try  to  generalize  the  notion  of  poly-1;ime 
many  one  and  poly-time  Turing  reduction  for  type  1  functions  to  the  type  2  case. 

Definition  6  A  functional  F{f,x)  many  one  reduces  to  a  functional  G{f,x),  if  there  are  basic  feasible 
functionals  (f>i,<hi4>3  such  that  F{f,x)  -  (^i(G(An^(/,x,n),^3(/,x)),/,x).  The  generalization  of  this 
notion  to  the  case  where  F  and/or  G  have  more  than  one  function  argument  etc.,  is  obvious. 

Definition  7  A  functional  F  Turing  reduces  to  a  functional  (or  function)  G,  if  F  can  be  represented 
by  a  X  term  constructed  from  G  using  basic  feasible  functionals  and  A  calculus  operations. 

A  machine  characterization  of  this  reducibility  can  be  obtained  by  machines  described  in  [12]  to 
compute  type  3  functionals.  Specifically,  F  Turing  reduces  to  G  iff  F  can  be  computed  by  a  k  count 
tape  HPOTM  which  uses  only  finitely  many  parameterizable  functionals  for  querying  and  whose  all 
type  2  inputs  have  been  fixed  to  G. 

We  shall  use  a  weaker  notion  than  Turing  reduction  to  define  a  version  of  polynomial  hierarchy. 
The  idea  of  this  weaker  Turing  reduction  is  that  if  F  reduces  to  G  than  F  can  be  computed  at  input 
/,x  using  G  as  oracle  such  that  whenever  G  is  queried  during  this  computation  its  type  1  input  is  a 
basic  feasible  function  in  /.  This  reduction  is  weaker  than  Turing  reduction  as  it  does  not  allow  G  to 
be  queried  at  type  1  input  which  is  a  basic  feasible  function  in  G,  f. 
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Definition  8  F  weakly  Turing  reduces  to  G  iff  there  is  a  k  count  tape  HPOTMM,  designed  to  compute 
a  type  3  functional,  such  that  it  uses  finitely  many  parameterizable  functionals,  none  of  which  takes  type 
2  inputs  of  M  as  parameters,  for  querying.  Further  M  computes  F  when  all  of  its  type  2  inputs  have 
been  fixed  to  G. 

It  is  correct  to  say  that  reductions  defined  above  are  reducibilities  as  they  are  reflexive  and  transi¬ 
tive.  All  the  reducibilities  defined  above  differ  on  the  space  of  computable  type  2  functionals,  Turing 
reducibility  is  the  most  general  and  many-one  reducibility  is  the  least  general.  A  many-one  reduction 
is  a  very  special  case  of  Turing  reduction. 

Lemma  3  There  are  computable  functionals  F,  G  such  that  F  weakly  Turing  reduces  to  G  but  F  does 
not  many  one  reduce  to  G. 

Proof:  This  is  plausible  if  we  notice  that  a  many  one  reduction  can  query  G  only  at  one  input  but 
a  weak  Turing  reduction  can  query  G  at  a  several  inputs  before  outputting  the  value  of  F  on  a  given 
input. 

Let  G  be  a  type  2  functional  of  type  [N  N] N  -*  N. 

Define  function  /  as  follows: 

f{x)  =  1  if  both  G{Xx.x,  x)  and  G{Xx.x  -I- 1,  x)  are  odd  or  both  are  even. 

=  0  otherwise 

.  /.  weakly  Turing  reduces  to  G,  for  every  G.  However  a  computable  G  can  be  designed  easily  by 
diagbnaiizing  over  all  triples  of  basic  feasible  functionals  such  that  /  does  not  many  one  reduce  to  G. 
G  is  designed  as  follows. 

Inputs  to.  G  are  g,  n. 

For  all  g,n  such  that  p(l)  ^  {1,2},  G{g,n)  is  defined  to  be  1. 

For  other  cases  G  is  constructed  in  stages,  each  stage  corresponding  to  a  natural  number. 

(with  stage  there  is  a  constant  Ui  associated,  ni  =  1) 
stage: 

Let  triple  of  BFF  be  <  ^i,  ^3  >. 

1.  Case  G(An.^(ji,nt),^3(ni))  is  defined  : 
if  <^i(G(An.^(n,ni),^3(ni)),ni)  =  0 

Iken  define  G{g,ni)  =  1  whenever  ^^(1)  G  {1,2} 
else  define  G{g,  ni)  —  p(l)  whenever  p(l)  G  {1, 2}. 

2.  Case  ^(l,ni)  =  1  : 

Define  G{Xn.4>2{n,ni),(f>z{ni))  =  1  and  evaluate  0i(l,ni) 

»/^i(l,ni)=0 

then  define  G{g,ni)  —  1  whenever  ^(1)  G  {1,2} 
else  define  G{g,ni)  =  ^(1)  whenever  g{l)  G  {1, 2}. 

[Notice  that  these  assignments  to  G  are  consistent  with  the  earlier  assignments  to  it.] 

3.  Case  ^^(l,ni)  =  2  : 

Define  G(An.^(n,ni),^3(ni))  =  1  and  evaluate  0i(l,ni) 
i/(^i(l,ni)  =  0 

then  define  G(g,ni)  —  1  whenever  p(l)  G  {1, 2} 
else  define  G{g,ni)  =  ^(l)  -  1  whenever  ((-(I)  G  {1, 2}. 


Let  nj+i  =  T7iax{ni,  (fsini)}  +  1. 

Define  G{g,x)  =  1  if  p(l)  G  {1,2}  and  x  <  Ui+i  and  G{g,x)  is  not  defined  so  far. 

End  of  stage  t  . 

It  is  not  difficult  to  see  that  the  above  construction  yields  a  well  defined  and  total  computable 
functional  G.  In  the  stage  we  made  sure  that  /  does  not  many  one  reduce  to  G  via  the  triple  of 
the  BFFs  .  Therefore  /  does  not  many  one  reduce  to  G.  □ 

Turing  reduction  is  more  general  than  the  weak  Turing  reduction  will  be  shown  in  section  7. 

For  generating  the  polynomial  hierarchy,  we  need  to  iterate  using  a  nondeterministic  operator. 
Therefore  we  define  a  nondeterministic  version  of  the  weak  Turing  reduction. 
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Definition  9  F,  a  0  —  1  valued  functional,  weakly  nondeterministic  Turing  reduces  to  G  iff  there  is  a 
nondeterministic  k  count  tape  HPOTM  M,  designed  to  compute  a  type  3  functional  such  that  it  uses 
finitely  many  parameterizahle  functionals,  none  of  which  takes  type  2  inputs  of  M  as  parameters,  for 
querying.  Further  M  computes  F  if  all  its  type  2  inputs  have  been  fixed  to  G. 

In  section  7  we  define  a  polynomial  hierarchy  using  weak  nondeterministic  Turing  reduction.  We  do 
not  define  a  nondeterministic  version  of  Turing  reduction  as  it  generates  a  hierarchy  which  collapses  at 
the  first  level.  This  will  be  shown  in  section  7. 


6  Victor  Harnik’s  type  2  polynomial  hierarchy 

In  [9],  Victor  Harnik  used  a  notion  of  polynomial  hierarchy  for  all  finite  types  in  order  to  extend  the 
results  of  [5].  In  this  section  we  study  the  type  2  section  this  hierarchy  in  some  detail.  Buss  has  also 
defined  a  similar  hierarchy  in  [3],  although  using  Godel  numbering. 

Definition  10  [9]  All  functionals  in  the  A  closure  of  BFF  and  where  Ff  is  the  i^^  level  of 
the  type  1  polynomial  hierarchy. 

This  hierarchy  does  not  specify  the  X'f ilf  ^  levels  explicitly.  For  the  sake  of  completeness  we  can 
take  =  rf  and  =  TTf. 

The  most  important  property  of  this  hierarchy,  which  is  required  in  proving  the  results  of  [9],  is  that 
it  preserves  O?.  That  is,  all  type  1  functions  constructed  using  functionals  in  functions  in  Df  and 
the  A  calculus  operations  are  in  .  In  fact  a  slightly  more  general  property  can  be  proved: 

Theorem  3  Let  C  he  a  complexity  class  of  type  1  functions  which  contains  P?  and  is  closed  under  com¬ 
position.  IfC  can  be  expressed  asC  —  \JiDTIME{ti{n)),  where  t'^s  are  monotone  and  time  constructible 
functions  then  C  is  preserved  by 

Proof:  Similar  to  that  of  theorem  3,  [11].  □ 

Observation:  The  above  type  2  hierarchy  collapses  iff  the  type  1  polynomial  hierarchy  collapses. 

This  is  because  if  type  1  poly  hierarchy  collapses,  say  Pf  =  Pj^i  then  by  definition  P^^^  - 
hence  the  given  type  2  hierarchy  iso  collapses.  On  the  other  hand,  if  /  e  Pf+i  -  Pf  then  /  €  Dj.t+i  “*^2,1 
also,  as  all  type  1  functions  in  P^^^  are  in  P? . 

It  is  possible  to  give  a  machine  characterization  of  this  class  using  count  tape  machines  of  [10]  which 
define  Ci  or  using  the  characterization  of  basic  feasible  functionals  by  Cook  and  Kapron  in  [6]. 

Lemma  4  A  functional  F  €  P2  iff  it  can  be  computed  by  some  count  tape  machine  M  of  class  Ci 
as  in  [10]  where  some  type  1  inputs  of  M  may  be  fixed  by  Pj^i  functions. 

Proof:  The  proof  is  similar  to  the  proof  of  lemma  6,  [11]  and  follows  from  the  definition  of  p2,t+i*  ^ 

Since  the  output  size  of  Pf  is  bounded  by  a  polynomial  it  is  possible  to  characterize  P^^^  using 
second  order  polynomials.  We  state  this  below  without  proof. 

Lemma  5  A  functional  is  in  P^  ^  iff  it  can  be  computed  by  a  type  2  oracle  TM  using  a  i7f  complete 
set  as  oracle  in  time  bounded  by  a  second  order  polynomial  in  the  length  of  type  0  and  type  1  inputs  to 
the  functional. 

Each  Pj  i  a  has  complete  problem,  namely  Ff  complete  problem  (which  can  also  be  thought  of  as 
a  type  2  functional)  under  both  Turing  and  the  weak  Turing  reduction.  The  two  reductions  coincide  as 
we  are  reducing  to  a  type  1  function.  Do  Pj^^  have  many  one  complete  problems?  A  negative  answer 
implies  P  ^  NP. 

We  can  define  another  type  2  poly-hierarchy  whose  level  is  obtained  by  taking  the  A  closure  of 
functionals  of  class  C2  with  P^,  this  hierarchy  also  preserves  the  P?  functions.  This  leads  to  a  somewhat 
informal,  general  conjecture. 
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Conjecture:  Let  C  be  the  largest  class  of  feasible  functionals  with  all  “desirable  preservation 
properties”  then  the  type  2  poly-hierarchy  obtained  by  taking  the  A  closure  of  functionals  of  C  with 
the  corresponding  level  of  type  1  poly  hierarchy  functions  yields  the  largest  hierarchy  with  all  the 
“desirable  preservation  properties”. 

7  Type  2  polynomial  hierarchy  PH’ 

In  this  section  we  define  another  version  of  polynomial  hierarchy  for  type  2  relations.  Unlike  the  hierarchy 
of  the  previous  section  levels  of  this  hierarchy  are  closed  under  appropriate  second  order  polynomial 
bounded  quantification.  This  hierarchy  has  two  natural  characterizations,  machine  based  and  quantifier 
based. 

We  first  introduce  a  simple  machine  model  to  define  this  hierarchy,  then  present  its  alternative 
characterizations  and  study  some  of  its  properties-such  as  existence  of  complete  sets  for  various  levels. 

A  nondeterministic  polynomial  time  oracle  Turing  machine  (NPOTM)  is  a  nondeterministic  oracle 
Turing  maehine  such  that  the  computation  time  on  each  path  is  bounded  by  a  fixed  polynomial  in 
the  length  of  the  maximum  of  the  largest  type  0  input  and  the  largest  answer  returned  on  that  path. 
Further,  computation  on  each  path  is  required  to  terminate. 

For  defining  our  version  of  polynomial  time  hierarchy  we  need  to  introduce  relativized  oracle  Turing 
machines  (ROTM).  An  ROTM  computes  a  functional  relative  to  some  functional  of  same  arity  (that  is 
the  functional  with  the  same  number  of  type  0  inputs  the  same  number  of  type  1  inputs).  Functional 
oracle  is  accessed  on  a  special  functional  oracle  tape  by  writing  type  0  input  to  the  functional  on 
this  tape.  Function  inputs  for  both  functionals,  the  functional  being  computed  and  the  functional  being 
queried,  are  taken  to  be  the  same.  To  clarify  the  above  concept  we  give  an  example.  An  OTM  computing 
F{f,n)  relative  to  functional  G{g,m)  has  an  oracle  tape  through  which  function  /  is  accessed.  Apart 
from  this  the  machine  also  has  a  functional  oracle  tape  through  which  G  is  accessed  by  writing  type  0 
input  for  functional  G.  If  OTM  computing  F  is  running  on  input  /,  n  and  queries  G  by  writing  x  on 
functional  oracle  tape  during  the  computation  then  the  value  of  G(/,  a;)  appears  on  the  functional  oracle 
answer  tape.  In  our  case  all  functional  oracles  will  be  relations.  RJPOTM  and  RNPOTM  are  defined  in 
an  obvious  manner.  Let  NP’  be  class  of  all  functionals  computable  by  NPOTMs  such  that  the  runtime 
on  each  path  is  bounded  by  a  second  order  polynomial. 

We  can  now  define  polynomial  hierarchy  for  type  2  in  analogy  with  polynomial  hierarchy  for  type  1 
as  follows.  We  denote  the  levels  of  this  hierarchy  with  the  usual  symbols  for  polynomial  hierarchy,  but 
with  superscript  ^ 

P'  =  basic  feasible  functionals  = 

K+i  —  ~  functionals  computable  by  RKPOTMs  in  second  order  polynomial  time  with 

relations  from  as  oracle. 

—  functionals  computable  by  RPOTMs  in  second  order  polynomial  time  with  relations 
from  as  oracle. 

PH>  =  Un>ir; 


By  the  argument  of  lemma  1  all  the  JT*^  classes  are  different.  This  argument  is  applicable 
because  when  all  type  1  inputs  are  fixed  to  a  0-1  valued  set  A,  then  the  classes  in  the  above  hierarchy 
are  the  corresponding  type  1  complexity  classes  relativized  with  oracle  A.  Oracles  that  separate  type  1 
polynomial  hierarchy  are  known  to  exist. 

E'n  (Un)  levels  of  this  version  of  polynomial  hierarchy  are  easily  seen  to  be  closed  under  second 
order  polynomial  bounded  existential  (universal)  quantification. 

Lemma  6  A  functional  is  in  E^.  iff  it  can  be  expressed  by  quantifying  a  basic  feasible  functional  by 
second  order  polynomial  bounded  quantifiers  with  at  most  k  alternations  of  quantifiers  such  that  the 
formula  begins  with  an  existential  quantifier.  A  functional  is  in  ITj^  iff  it  can  be  expressed  by  a  formula 
as  above  except  that  it  must  start  with  a  universal  quantifier  now. 
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Proof:  Observe  that  for  each  second  order  polynomial  P(/,x)  there  is  a  type-2  NP  machine  which 
works  in  second  order  polynomial  time  and  on  input  /,x  computes  P(!/l,lx|)  on  some  branch  and  on 
all  other  branches  a  number  <  P(|/t,  |a:|). 

The  rest  of  the  proof  is  a  direct  generalization  of  the  characterization  theorem  for  polynomial 
hierarchy  of  type  1.  (see  proofs  of  theorems  8.2  &  8.3  in  [1]).  □ 

There  is  also  an  alternative  way  to  present  A'  levels  of  the  hierarchy  PH'  as  shown  in  the  next 
lemma.  For  terms  not  defined  here,  see  [14]. 

Lemma  7  For  n  >  0,  A'^^i  »s  the  class  of  functionals  obtained  from  functionals  of  class  by  appli¬ 
cation,  expansion,  functional  composition  and  limited  recursion  on  notation. 

Note  that  functional  abstraction  is  not  included  in  the  above  lemma. 

Proof  (sketch):  Let  F(f,n)  6  and  let  M  be  a  RPOTM  computing  F(f,n)  with  oracle  G(f,n), 
where  G(f,  n)  €  Prom  M  it  is  easy  to  design  a  POTM  M'  such  that  M'  computes  a  functional 
F'(f,  g,  n)  in  time  P(|/|,  [ffl,  Inj)  for  some  second  order  polynomial  P.  Further  F(f,  n)  =  P'(/,  AxG(/,  x),  n). 
By  [7],  F'  is  a  basic  feasible  functional.  By  using  the  same  reasoning  as  in  section  4,  [4]  it  can  be  con¬ 
structed  from  some  initial  functions  using  application,  expansion,  functional  composition  and  limited 
recursion  on  notation.  By  induction  on  the  length  of  such  derivation  of  P',  F'(/,AxG(/,x),n)  can  be 
shown  to  be  constructible  by  application,  expansion,  functional  composition  and  limited  recursion  on 
notation  from  the  functionals  in  □. 

We  now  turn  our  attention  to  the  existence  of  complete  sets  for  the  levels  of  hierarchy  PH'. 

Lemma  8  Let  C  be  a  class  of  type  2  relations,  such  that  C  is  contained  in  class  of  relations  obtained 
from  POTM  predicates  using  POTM  functional  bounded  existential  quantification.  If  C  contains  POLY 
and  is  closed  under  POLY  bounded  existential  quantification  then  C  has  no  complete  relation  under 
many  one  reduction. 

Proof:  Let  P(/,x)  €  C  be  a  complete  relation  for  C.  Then  Risk  quantifier  relation  for  some  k.  (that 
is,  R{f,  x)  can  be  represented  as  a  formula  obtained  from  a  POTM  computable  predicate  by  prefixing 
it  with  k  POTM  computable  bounded  quantifiers) 

We  use  the  similar  argument  as  in  lemma  2  to  derive  a  contradiction. 

We  can  demonstrate  a  A:  -}-  2  existential  quantifier  formula  A{f,  x)  which  does  not  many  one  reduce 
to  R  by  any  triple  basic  feasible  functionals. 

For  each  candidate  triple  <l>i,<f>2i  <l>3  of  basic  feasible  functionals  we  construct  a  function  /  such  that 
formula  <l>i{R{Xn<f>2{f,x,n),(l>3{f,x)),f,x)  is  true  iff  the  formula  A  is  false. 

,  The  construction  of  /  is  very  similar  to  the  construction  in  lemma  2,  so  we  omit  the  details  here. 

□ 

Corollary  4  For  all  k>l,  E'f.  does  not  have  any  complete  set  under  many  one  reduction. 

Proof  :  Using  same  idea  as  in  the  lemma  above,  □ 

Remark:  We  do  not  know  if  Tj.  have  complete  sets  under  feasible  Turing  reduction. 

Although  our  definition  of  RPOTMs  and  RNPOTMs  is  very  simple,  the  above  hierarchy  is  same  as 
the  one  generated  using  weak  nondeterministic  Turing  operator. 

Theorem  4  E'n+i=set  of  predicates  which  are  weak  nondeterministic  Turing  reducible  to  E^. 

Proof:  That  E^^i  contains  only  weak  nondeterministic  reducible  predicates  to  E|^  is  obvious  by  the 
definition  of  as  RNPOTMs  are  specialized  machines  which  witness  weak  nondeterministic  Turing 
reduction  to  type  2  functionals.  To  prove  the  other  direction  we  first  show  that  if  P(/,x)  e  El^  and 
are  BFF  then  H(f,x,u)  =  F{XyMfyX,y),Mf^^))  €  E'^.  This  can  be  proved  by 
induction  on  n.  For  the  base  case  let  F{f,x)  6  E'^,  the  machine  to  compute  H{f,x,u)  in  this  case  is 
designed  by  modifying  the  machine  M  which  computes  F(/,  x),  for  each  query  of  the  machine  M  to  f 
at  y  we  answer  the  query  by  running  ^i(/,x,y).  For  the  induction  step  let  F(/,x)  6  E'^+i  be  computed 
by  RNPOTM  M'  with  an  oracle  G{f,z)  €  r;.  Let  G'(/,x,z)  =  G{Xy<f>i{f,x,y),z),  by  induction 
hypothesis  G'  €  E'„.  Consider  a  RNPOTM  machine  M"  with  oracle  G'  which  on  input  /,  x,  u  starts 
simulating  M'  with  type  0  input  as  <f>2{f, u),  M"  answers  query  of  M'  to  type  1  input  at  argument  m  as 
<f>i{f,x,  m).  A  query  of  M'  to  type  2  oracle  at  argument  m  is  converted  by  M"  to  a  query  at  argument 
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X,  m  to  its  type  2  oracle.  It  is  easy  to  see  that  M"  vith  its  type  2  oracle  as  computes  H{ft  x,  u)  and 
can  be  made  to  work  in  time  polynomial  in  |/|,  |x|,  |wj.  Hence  H  € 

Now  let  some  machine  Mi  weakly  nondeterministic  Turing  reduce  a  set  A  to  functional  F(/,  x)  G 
Since  Mi  uses  only  finitely  many  query  functionals,  Mi  can  be  thought  of  querying  functionals 
obtained  by  the  construction  of  the  preceding  paragraph,  at  function  input  /  only. 
Therefore  Mi  can  be  modified  to  an  RNPOTM  computing  set  A  relative  to  functionals  F/, . . . ,  F/  in 
second  order  polynomial  time.  □ 

Theorem  5  U„>oi;;+i  C  BFF{S[) 

Proof:  The  idea  is  similar  to  that  in  the  proof  of  lemma  1.  We  just  give  an  example.  We  show  that 
A{f,z)  -3x<  <  ^(/,z,x)[G(/,x,j/,2:)],  where  0i,<A2,G  are  BFF  is  always  in  BFF(i7(). 

Let  functional  F(/,  g, z)-3x<  (f>i (/,  z)[g{z, x)  #  Ij,  F  €  . 

Let  be  the  characteristic  function  of  predicate  3y  <  <f>2{f,  z,x)[-^G{f,x,y,  z)]. 

H(/,z,x)€ri. 

Now  F(/,AzAxF(/,z,x),z)  -3x<  <^i(/,z)Vy  <  ^(/,2:,a;)[G(/,x,y,z)]  ~A{f,z). 

Therefore  A(/,  z)  6  BFF(i:0. 

By  repeating  this  argument  it  is  easily  shown  that  UnSiO-E'n+i  Q  FFF(X'{).  □ 

Corollary  5  Turing  reduction  is  more  powerful  than  weak  Turing  reduction. 

Proof:  Since  ^  X'j,  by  theorem  4  it  follows  that  there  is  a  Functional  in  X3  which  does  not  weak 
Turing  reduce  (in  fact  it  does  not  even  weak  nondeterministic  Turing  reduce  )  to  XJ.  However  by 
previous  theorem  every  functional  in  X3  Turing  reduces  to  X( .  □ 

8  PH’  equals  polynomially  bounded  relations 

Though  each  level  of  PH*  is  larger  than  the  corresponding  level  of  the  class  of  polynomially  bounded 
relations  of  section  3,  the  following  theorem  shows  that  both  contain  the  same  relations. 

Theorem  6  PH*  is  contained  in  the  class  of  polynomially  bounded  relations. 

Proof:  Let  R  be  a  relation  in  PH’.  R  can  be  represented  as  a  basic  feasible  functional  prefixed  by  a 
string  of  second  order  polynomial  bounded  quantifiers.  We  can  obtain  a  relation  equivalent  to  R  by 
replacing  each  second  order  quantifier  by  a  string  of  POLY  bounded  quantifiers,  as  indicated  below. 

Observe  that  any  second  order  polynomial  can  be  simulated  by  a  string  of  POLY  length  bounded 
quantifiers.  To  illustrate  by  an  example  let  the  second  order  polynomial  be  P(f,  x)  =  /(/(x^  )+x^)+X''  . 

3iz|  <  F(j/|,  |x|)  is  equivalent  to  3|xi|  <  |xp3|x2|  <  |/(xi)|  +  ixp'3|z|  <  |/(x2)|  +  |xp‘. 
in  the  sense  that  the  former  can  be  replaced  by  the  latter  in  any  formula  (in  which  the  variables 
xi ,  X2  do  not  appear)  without  changing  the  meaning  of  the  formula. 

In  the  same  manner  V|z|  <  F(|/|,  tx|)  is  equivalent  to 
V|ll|  <  |x|'V|xj|  <  |/(x,)l  +  kNx|  <  |/(X2)|  +  \x\l. 

After  replacing  each  quantifier  by  the  process  indicated  above  we  obtain  a  relation  equivalent  to  R 
which  involves  only  POLY  bounded  quantifiers.  R  is  therefore  in  the  class  of  polynomially  bounded 
relations. 

General  construction  of  POLY  bounded  quantifier  strings  for  simulating  second  order  polynomials 
(with  length  of  type  0  and  type  1  objects  as  input  to  the  polynomial)  can  be  defined  inductively  along 
with  the  definition  of  second  order  polynomials.  □ 


9  Conclusion 

In  this  paper  we  have  shown  limitations  on  finding  a  faithful  type  2  analog  of  type  1  poly-hierarchy.  We 
then  studied  two  polynomial  hierarchies  which  seem  natural  from  different  viewpoints.  The  hierarchy 
of  section  6  is  properly  contained  in  the  hierarchy  PH\  so  perhaps  Victor  Harnick’s  hierarchy  may  be 
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called  “basic  polynomial  hierarchy”.  However  we  need  to  figure  out  what  the  key  criteria  for  defining 
type  2  poly-hierarchy  should  be  to  make  some  progress.  The  subject  of  feasible  reduction  between  type 
■  2  functionals  also  requires  further  study. 

We  also  solved  niost  of  problems  posed  in  the  work  of  Townsend  [14].  Townsend  [14]  suggested 
the  use  of  topological  concepts  for  solving  these  problems  but  we  solve  them  without  involving  such 
concepts.  In  fact,  we  can  obtain  some  results  about  open  sets  introduced  in  [14]  from  our  work.  For 
instance  A  €  POLY  iff  A  is  fPOLY.  POLY.  POLYl  open  iff  A  is  rPOLY.  POLY.  POLYl  closed. 
However  topological  concepts  may  be  useful  in  studying  fine  structure  and  interrelationships  among 
type  2  complexity  classes.  It  should  be  possible  to  examine  a  hierarchy  using  the  function  quantifiers. 
This  can  give  a  feasible  analog  of  analytical  hierarchy,  which  perhaps  can  be  used  in  effective  descriptive 
set  theory. 
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The  Hierarchy  of  Terminating  Recursive 
Programs  over  N 

Stanley  S.  Wainer  * 

Pure  Mathematics  Dept.,  Leeds  University,  Leeds  LS2  9JT,  UK 


1  Introduction 

A  terminating  recursive  program  defines  a  total  recursive  functional,  taking 
“given”  functions  to  the  function  defined  from  them  by  the  program.  Termi¬ 
nation  means  that  the  program  has  a  well-founded  computation  tree  with  a 
recursive  ordinal  as  its  height,  and  Kleene  (1958)  noted  that,  in  contrast  with 
the  well  known  “collapsing  phenomenon”  for  hierarchies  of  recursive  functions, 
the  resulting  hierarchy  expands  right  the  way  through  wf :  i.e.  for  each  recur¬ 
sive  ordinal  a  there  is  a  total  recursive  functional  which  cannot  be  defined  by 
any  program  of  height  less  than  a. 

In  this  paper  we  examine  some  of  the  ways  in  which  the  ordinal  height  of 
a  program  encodes  its  complexity.  By  a  careful  assignment  of  (proof  theoretic) 
ordinal  bounds  to  derivations  in  Kleene’s  equation  calculus,  the  standard  “fast” , 
“medium”  and  “slow”  growing  hierarchies  emerge  as  canonical  complexity  mea¬ 
sures  allowing  different  forms  of  recursion  to  be  classified  and  compared.  Known 
relationships  between  these  hierarchies  then  yield  measures  of  “transformational 
complexity”  (e.g.  recursive  to  tail  recursive)  in  terms  of  their  corresponding 
ordinal  trade-offs.  The  underlying  theme  is  that  of  Cut  Elimination,  but  in  an 
equational  setting. 

2  Kleene’s  Hierarchy  of  Recursive  Functionals 

A  recursive  program,  defining  a  (possibly  partial)  function  /  :  JV*  — *  N  from 
“given”  functions  . . .  ,pm)  can  be  viewed  as  a  monotone,  continuous  operator 
$  on  partial  functions,  having  /  as  its  least  fixed  point 

/  =  . 

*  Partially  supported  by  an  EU  Science  Plan  Twinning  Project  between  Leeds,  Munich 
and  Oslo,  entitled  “Proof  Theory  and  Computation”.  The  author  thanks  D.  Leivant  and  H. 
Schwichtenberg  for  valuable  comments  on  an  earlier  draft. 
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To  simplify  notation,  we  shall  regard  the  given  functions  as  having  been  coded 
into  a  single  unary  function  g{z)  -  {gi{z), . . .  and  similarly  the  argu¬ 

ments  of  /  as  having  been  coded  into  a  single  argument  x  —  {xi, . . .  ,Xk).  Thus 
the  fixed-point  equation  defining  /  is  just 

f{x)  =  ^gj){x)  . 

The  solution  to  this  equation  is 


f  =  [jfi 

i 

where  =  $(^)‘(X)  =  ^{g, . , .  $(5,  ±)) . . .)  so  that  fi  C  for  every 

i  =  0, 1, 2, . . .  Therefore  for  all  n^m  e 

/(n)  =  m  ^  {fi{n)  ~  m) 

and  since  the  successful  evaluation  of  fi(n)  =  $(p)*(±)(72)  will  require  only 
finitely  many  calls  on  g,  say  ^(0), p(l), . . . ,  - 1),  we  can  go  a  step  further  and 

write 

/(n)  =  m  ^  3j  ($(^(j))*(X)(n)  =  m) 

where  g{j)  denotes  the  finite  subfunction  of  g  from  ^(0)  up  to  g{j  —  1).  Further¬ 
more  since  $  is  monotone  we  can  contract  the  two  existential  quantifiers  into 
one  by  taking  any  y  >  max(i,  j)  thus 

/(n)  =  m  ^  3y  ($(^(2/))^(X)(n)  =  m)  . 

^Prom  this  we  can  immediately  derive  Kleene’s  fundamental  Normal  Form  for  re¬ 
cursion.  Below  we  let  cr,  r  range  over  the  set  of  all  finite  sequences  (ico,  •  •  • , 
of  natural  numbers,  so  that  cr  =  g(j)  means  g{i)  =  Xi  for  each  i  <  j.  We  let  lha 
denote  the  length  of  a  z  denote  the  sequence  obtained  by  appending  z  to 
the  end  of  a,  and  a  ^  r  denote  the  new  sequence  obtained  by  concatenating 
the  two  etc. 

Theorem  2.1  Let  U  be  any  recursive  surjection  from  N  to  N  such  that  U~^{n) 
is  infinite  for  every  n.  Then  with  each  operator  $  as  above,  we  can  associate  a 
recursive  relation 

T^(n,a,y)  :=  $((T)>'(X)(n)  1=  17(3/) 

such  that  if  f  is  recursively  defined  from  a  given  function  g  as  the  least  fixed 
point  of  the  equation 

f  = 

then  f  is  expressible  in  the  normal  form 


/(n)  =  U{  least  y.  T<t{n,g{y),y) )  . 
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Remark.  7$  would  normally  be  primitive  recursive.  Furthermore  if  we  pa¬ 
rameterized  T  by  introducing  a  new  variable  e  ranging  over  codes  for  operators 
then  it  would  become  a  version  of  Kleene’s  “T-predicate” . 

Definitions.  With  each  operator  $  as  above,  associate  the  following  partial 
recursive  functional  from  partial  functions  g  to  partial  functions  /, 

F^{g)  :=  the  least  fixed  point  /  of  the  equation  /  =  /)  . 

Call  $  a  terminating  operator  and  a  total  recursive  functional  if  for  each 
totally  defined  function  g  :  N  ^  N  the  output  function  /  =  (g)  is  also  totally 

defined. 

Theorem  2.2  The  normal  form  for  is 

F*{9)(n)  =  U{  least  y.  Ti{n,g{y),y) ) . 

and  therefore 

is  a  total  recursive  functional  Wg  Vn  3y  Ti^{n,g{y),y)  . 

Definition.  With  each  operator  $  as  above,  associate  the  following  tree  of  finite 
sequences  of  natural  numbers  : 

Tree($)  :=  {  {so, . . .  ,xt)  :  Vj/ <  fc -ir*(a;o,  {si,. .  .,2:,),3/)  }  . 

Tree($)  is  called  the  computation-tree  of  $. 

We  imagine  Tree($)  as  growing  downwards  by  extension,  i.e.  if  a  and  r  are 
any  two  nodes  on  the  tree  then  cr  comes  below  r  iff  cr  is  an  extension  of  r.  An 
infinite  branch  of  the  tree  is  thus  determined  by  a  number  n  and  a  total  function 
g  :  N  N  such  that  >fy  -’T$(n,  g{y),  y).  Therefore  an  infinite  branch  on  Tree($) 
is  a  witness  to  the  fact  that  for  some  n  and  p,  F$(p)(n)  is  not  defined.  To  say 
that  the  tree  is  “well-founded”  is  to  say  that  there  are  no  infinite  branches,  and 
hence  : 

Theorem  2.3 


$  is  terminating  ^  Tree{^)  is  well-founded. 

This  equivalence  is  the  basis  for  a  natural  theory  of  ordinal  assignments.  It 
allows  us  to  assign  to  each  terminating  operator  $  a  (recursive)  ordinal  ||$|| 
measuring  the  size  of  its  computation- tree. 

Definition.  For  each  node  a  on  Tree($)  define 

||o-||  =0  if  (7  is  a  leaf 

=  sup  {  ||r||  +  1 :  r  extends  cr  }  otherwise. 
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Then  define  ||#||  :=  ||()||  where  (}  is  the  empty  sequence. 

Example.  The  iteration-from-O  operator 

/)(^)  •=  if  n  =  0  then  0  else  g{f{n  —  1)) 
computes  the  total  recursive  functional 

F^{9){n)  =  9-{0)  . 

The  terminal  node  or  “leaf”  on  the  branch  through  Tree($)  determined  by  n 
and  9  will  be  the  shortest  sequence  of  the  form  (n,5(0),  ...,9(9-  1)}  such  that 
p^(0)  <  y  for  every  i  <  and  U{y)  —  p”(0).  In  particular,  if  9  is  such  that 
5®(0)  <  for  every  i  <n  then  this  sequence  will  be  of  the  form 

(n,ff(0), . . . ,s'(0), . . . , <7^(0), . . . ,p”-i(0), . . .  ,g"(0), ...,g{y-  1)> 

where  the  value  of  ^^(0),  once  fixed,  determines  the  length  of  the  ensuing  segment 
up  to  ^*■*■^(0).  Therefore  as  we  take  the  supremum  over  all  branches  issuing  from 
a  given  node 

the  successive  segments  5*(0), . .  .,9^'^^{0)  have  unbounded  length,  depending  on 
the  value  of  9^(0).  So  each  such  segment  adds  one  more  a;  to  the  ordinal  height 
of  the  tree.  Since  there  are  n  such  segments,  the  height  of  the  subtree  below 
node  (n)  must  be  a;  •  n.  Therefore  the  height  of  the  entire  tree  is 

||$||  =  sup  U)  '71  =  . 

n 

Definition.  For  each  recursive  ordinal  a  define 

REC2{ol)  =  {  :  $  terminating  and  11$ II  <  a}. 

Kleene  (1958)  noted  that,  in  sharp  contrzist  with  any  inductive  hierarchy 
for  all  recursive  functions,  this  hierarchy  of  total  recursive  functionals  does  not 
depend  on  any  system  of  ordinal  notations  and  furthermore  it  does  not  collapse. 

Theorem  2.4  For  each  recursive  ordinal  a  there  is  a  total  recursive  functional 
which  does  not  belong  to  i2EC2(a). 

Proof.  Assume  a  is  a  fixed  recursive  ordinal  and  choose  a  recursive  well¬ 
ordering  with  that  order  type.  Choose  also  a  Godel  numbering  of  all  partial 
recursive  operators  $  e.  Define  a  recursive  functional  /,5,n,cr)  as 
follows, 


Kt(e,/,p,n,cr)  =  if  ~>T^(n,a,lha)  and 

f({n)  ^  gilha))  -<oc  f{{n)  a) 

then  Va{e,f,g,n,a'--g{lha)) 
else  U{lh(T)  . 
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Notice  that  the  well-foundedness  of  ensures  that  Va  is  total.  Furthermore  if 
II $11  <  a  then  there  will  be  an  order-preserving  map  /  :  Tree($)  — ►  a  so  that 
for  all  g  and  n, 

Va(e,f,g,n,{))  =  F*(p)(n)  . 

Consequently  the  total  recursive  functional  F  defined  by 

F{9){n)  =  V<.(ff(0).Ax.<?(a  +  l),<,,n,())  +  l 

cannot  lie  in  REC^ict)-  For  if  it  did  there  would  be  an  operator  $  with  Godel 
number  e  defining  it,  and  an  order-preserving  map  /  :  Tree($)  — ^  a,  so  that  for 
the  fixed  function  argument  g  with  ^(0)  =  e  and  g{x  +  1)  =  f{x)  we’d  have 

F(9){n)  =  Va(e,f,g,n,())  +  l  =  F(p)(n)  +  1 

a  contradiction. 


3  Recursion  vs  Tail  Recursion 

How  does  the  ordinal  ||$||  reflect  the  “complexity”  of  $  ? 

Definition.  Associate  with  each  terminating  operator  $  the  functional 

:  Tree($)  ->  {N N)  {N  ^  N) 

defined  by  the  following  tail  recursion  over  nodes  a  on  Tree($)  : 

{(T){g){m)  =  m  if  a  is  a  leaf 

=  (a  ^  g{m)){g){7n  +  1)  otherwise. 

Theorem  3.1  For  each  terminating  operator  $  we  have 

Hif  ((n))(ff)(0)  =  least  y.  Ti{n,g{y),y) ) . 

Tf)  PTPfnvp 

F^{g){n)  =  t/(  ((n»(p)(0) ) . 

Therefore  m  =  F$(p)(7i)  can  be  computed  by  the  following  abstract  ” while 

program”  over  Tree(^) 

a  :=  (n)  ; 
m  :=  0  ; 

while  a  is  not  a  leaf  do  tr  :=  a  g{'m)  ;  *=  m  4*  1  od  ; 

m  :=  U{m)  . 

Proof.  Let  j/o  =  least  y.  T^{n,g{y)^y).  Then  from  the  definition  above  we 
compute 

i{n)){gm  =  ((n,g(0)»(<?)(l) 

=  H4  {{n,gi0),g{l))){g)(2) 

=  ((n,5(0),5(l),5(2)»(<?)(3) 

=  ((n,ff(0),...,ff(2/o  -  l)))(5)(2/o) 

=  yo  ■ 


286 


The  rest  follows  immediately  from  the  Normal  Form  for  and  from  the  direct 
translation  of  the  tail-recursive  definition  of  into  a  while-loop. 

Definition.  Call  a  terminating  operator  and  its  associated  recursive  func¬ 
tional  a-recursive  where  a  is  an  ordinal,  if  there  is  a  function  |.|  :  N"  ^  a 
such  that  for  all  g  and  n,  the  recursive  calls  on  /  made  by  $  in  evaluating  the 
right  hand  side  of 

f(n)  = 

are  always  of  the  form  /(m)  where  |m|  <  |n|. 

Example.  The  previous  example  of  an  iteration  operator  $  is  cj-recursive  since 
we  can  simply  take  |n|  =  n. 

Lemma  3.2  If  ^  is  a-recursive  then  ||$||  <u^. 

Proof.  Suppose  /  is  defined  from  a  given  function  g  by  a-recursion  thus 

f{n)  =  ^{gj){n). 

For  any  fixed  n  let  /(no),  /(ni),  ...  ,  /(nj),...  be  the  finitely-many  recursive 
calls  used  in  evaluating  the  right  hand  side,  so  \ni\  <  |n|  <  a  for  each  i.  Note 
that  in  general  there  may  be  nested  calls,  meaning  that  the  value  of  ni  might 
depend  upon  some  other  /(7ij)’s. 

In  the  computation  tree  Tree($),  all  of  these  recursive  calls,  and  the  succes¬ 
sive  calls  needed  for  their  evaluations  etcetera,  must  be  arranged  linearly  along 
the  ^-branch  below  node  {n). 

What  we  are  looking  for  is  a  function  ^  on  ordinals  such  that 

ll(n)||  <  ^(H). 

But  from  the  above  comment  we  see  that  ijj  must  satisfy  the  condition  : 
i^{\n\)  <  T.i'ipilnil)  <  max'0(|nf|) -u; 

I 

and  so  the  obvious  solution  is 


^(H)  =  0)1”!  . 

Therefore 

ll(n)||  < 

and  hence,  taking  the  supremum  over  all  nodes  (ti)  in  Tree($),  we  obtain 

m\  <  . 

Note.  If  in  the  above  Lemma,  there  were  a  fixed  bound  k  on  the  number  of 
recursive  calls  on  /  induced  by  #,  then  the  ip  function  need  only  satisfy  : 

V>(|ti|)  <  V’(ko|)  +  ^fr(|ni|)  +  ...4-^(|n*_i|)  <  maxV'(|ni|)  •  fc 


287 


and  so  we  could  then  choose  a  better  bound  : 

tPi\n\)  =  fcl"l 

in  which  case 

m  <  • 

Combining  the  Lemma  with  the  previous  Theorem,  we  thus  have  an  “expo¬ 
nential  trade-off”  in  the  ordinal  complexity  of  the  transformation  from  general 
recursive  programs  to  tail  recursive  ones.  In  fact  the  result  below  goes  back 
originally  to  Tait(1961)  who  first  showed  how  arbitrary  nested  recursions  could 
be  transformed  into  unnested  ones.  But  it  was  re-examined  more  recently  in 
Fairtlough-Wainer(1992)  in  the  context  of  transformations  to  “while” -programs. 
The  proofs  there  are  different  from  Tait’s,  and  different  again  from  the  ones 
presented  here,  although  they  all  have  the  same  germ.. 

Theorem  3.3  Every  a-recursive  operator  $  can  he  transformed  into  an  w®- 
t ail-recursive  definition  of  such  that  for  all  g  :  N  N  and  all  n  G  N, 

F^{9)(n)  =  U(H^{(n)){gm). 

Furthermore,  if  there  is  a  fixed  bound  k  on  the  number  of  recursive  calls  in 
then  cj®  can  he  strengthened  to  k^. 

Example.  Let  $  be  the  operator  defining  a  binary  function  /  =  as 

follows  ; 

$(s,/)(n,m)  =  if  n  =  0  then  g{m)  else  f{n  —  1,  f{n  —  1,  m))  . 

Clearly  $  is  w-recursive  under  the  assignment  |(n,m)|  =  n  and,  writing  to 
denote  the  function  \m,f{n,m),  we  see  that  for  all  n,m  €  N, 

fn  M  =  fn-i  o  /„_2  O  . . .  o  /i  o  /o  ( g{m)  )  . 

Now  for  any  strictly  decreasing  sequence  of  numbers  a  =  (^i , . . . ,  ir)  set 

H{a){g)  :=  /i,  o  0  ...  o 

Then  for  all  n,  m  we  have 


fn  (m)  =  H  ((n))(g){m) 

and  furthermore,  H  is  definable  by  the  following  tail  recursion  : 

H  {()){g){m)  =  m 

H  (<r)(s)(m)  =  H  {T){g){g{m)) 

where  if  o’  =  {n, . . .  ,ir}  then 

.T  —  (ii, . . .  —  1,^7- —  2, . . . ,  1,0)  . 
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But  notice  that  this  tail  recursive  definition  of  is  a  2‘*'-recursion  under  the 
assignment 

\cr\  =  2*'  +  2*"  +  . . .  -h  2*"  . 

Thus  we  have  transformed  the  doubly-nested  cj-recursion  #  into  a  2‘*'-tail-  re¬ 
cursion. 

Remark.  Of  course  there  is  no  difference  set  theoretically  between  the  ordinals 
u  and  2^.  However  the  structure  of  their  respective  well-orderings  used  in  the 
above  example  is  quite  different.  It  is  the  increase  in  complexity  of  the  well¬ 
ordering  on  2^  which  reflects  the  cost  in  transforming  the  a;-recursive  $  to  a 
tail  recursion.  In  other  words  it  is  not  really  the  set-theoretic  notion  of  ordinal 
which  matters  here,  but  rather  the  “intensional”  representation  of  it  by  means 
of  a  chosen  well-ordering.  This  distinction  is  brought  out  further  in  the  next 
section. 


4  Majorization  Hierarchies 

As  noted  above,  an  ordinal  for  us  is  not  simply  a  set.  What  matters  is  the  way 
in  which  it  is  presented,  for  it  is  the  presentation  that  provides  a  structure  over 
which  we  can  make  recursive  definitions.  See  Buchholz,  Cichon  and  Weiermann 
(1994)  for  an  alternative  but  closely  related  treatment  of  the  ideas  below. 

Definitions,  (i)  By  a  presentation  of  a  countable  ordinal  a  we  mean  a  chosen 
sequence  of  finite  subsets  a[f]  C  a  satisfying  : 

a[0]  C  q:[1]  C  ...  C  a\i]  C  a[i-\-l]  C  ... 


and 

a  =  ya[j] . 

i 

We  also  insist  that  0  G  a[0]  for  all  a  >  0. 

(ii)  Note  that  a  presentation  of  a  induces  a  presentation  of  each  <ahy 

/9[i]  =  a[i]n/?. 

(iii)  Each  presentation  of  a  determines  a  sequence  of  predecessor  functions 
Pi  :  Q:  -I- 1  — ►  a  defined  by 

Pi(0)  =  0  and  Pi{/3)  =  max  I3[i]  ii  0  <  <  a  . 

Thus  if  k  is  the  cardinality  of  fi[i]  we  have  : 

m  =  {0 = ptm  <  pt\^)  <  <  ph^)  <  Pirn  ■ 

Definitions.  Given  fixed  presentations  of  a  and  the  predecessor  functions  Pi 
provide  a  natural  way  of  recursively  defining  presentations  of  the  sum,  product 
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and  exponential  as  follows  : 

(a  +  0)[i] 

=  a[i] 

(a +  /?)[*] 

=  (a  +  Fi(^))[ilU{a  +  Pi(W 

(a  ■  0)[i] 

=  0[i] 

(a  •  I3)[i] 

=  (a  •  Pi(^)  +  a)[i] 

=  m 

For  k  e  N  we  choose  the  presentation  k[i]  =  {0, 1, . . . ,  A;  —  1}  for  every  i.  Then 
if  we  give  cj  the  presentation  uj[i]  =  (i  +  l)[i]  =  {0, 1, . . . ,  i}  the  above  definitions 
yield  standard  presentations  for  all  ordinals  below  eo  =  sup{l,u;,ci;‘^,a;‘^  >•••}• 

Definition.  Let  F  :  (AT  ->  N)^  {N  ->  N)  be  any  total  functional,  let  a 
be  any  fixed  countable  ordinal  with  a  chosen  presentation,  and  let  h^g  be  given 
functions.  Then  the  hierarchy  generated  by  iterating  F  over  a  consists  of  the 
functions  fpiN^N  where  13  <  a,  defined  as  follows  : 

/o(n)  =  h{n) 

fp{n)  =  F{g,  /p„(/3))(n)  if  0  <  /3  <  a  . 

Lemma  4.1  Suppose  the  functional  F  satisfies  the  following  “majorisation  con¬ 
ditions”  when  restricted  to  strictly  increasing,  positive  functions  g  and  f  : 

f{n)  <  F{gJ)in)  <  F{gJ){n  +  l) 

and 

Vm  >  n  (/(m)  < /(m))  F{g,f){n)  <  F(g,f){n). 

Suppose  also  that  h  and  g  are  strictly  increasing  and  that  g  is  positive. 

Then  the  hierarchy  {fp  :  (3  <  a}  generated  by  F  satisfies  : 

•  7  e  f3[n]  /^(n)  <  fpin). 

•  fp  is  strictly  increasing. 

Proof.  By  inductions  over  <  a. 

If  7  €  fi[n]  then  either  7  €  Pn{fi)ln]  or  7  =  Pn{l3)  and  so  by  the  induction 
hypothesis  and  the  first  majorisation  condition  we  have 

/7W  <  /Pn(/3)W  <  =  fpM  • 

To  see  that  fp  is  strictly  increasing,  first  notice  that  either  Pn{l3)  ^  ■Fn+i(^)[w+l] 
or  else  Pnifi)  =  Pn+iW),  because  fi[n]  C  fi[n  +  1].  Therefore  by  the  induction 
hypothesis  we  have 


Vm  >  n  +  1  (fp„(0){m)  <  fp„+^{0){m))  . 


290 


So  by  the  induction  hypothesis  and  the  majorisation  conditions, 

fffin)  =F(3,/p„(^))(n) 

<Pi9ifp„{0)){n  +  l) 

<Pi9,fp„+i{0))(n  +  l) 

=  Mn  +  1)  . 

This  completes  the  proof, 

Majorisation  Hierarchies.  The  following  are  the  main  examples  of  hierarchies 
generated  by  the  above  Lemma  -  and  we  shall  make  use  of  them  later. 

•  Choosing  F{g,  f)  =  gofwe  obtain  : 

Go{9){n)  =  0  ;  G^{g){n)  =  g{  Gp^(i3)(g){n) ) . 

•  Choosing  F{g,  /)  =  /  o  p  we  obtain  : 

Hoi9)in)  =  n;  H0ig)in)  =  p(n)  )  . 

•  Choosing  F(p,  f)  =  fof  =  f^we  obtain  : 

Bo{9){n)  =  g{n)  ;  B^(p)(n)  =  Bp^^p){gf(n)  . 

These  are  called  respectively  the  “Slow  Growing”,  “Hardy”  and  (a  version  of 
the)  “Fast  Growing”  hierarchies.  When  g  is  chosen  to  be  the  successor  function 
we  merely  write  Gp{n),  Hp{n),  Bp{n)  instead  of  Gp{succ)(n),  Hp{succ)(n), 
Bp{succ){n).  Although  the  definition  of  G  does  not  quite  satisfy  the  conditions 
of  the  Lemma,  because  we  start  with  h  the  constant  zero  function,  nevertheless 
the  G  functions  will  be  strictly  increasing  after  the  first  limit  stage. 

Lemma  4.2  Gp{n)  ~  cardinality  of  ^[n]  . 

Proof.  By  a  simple  transfinite  induction  on  If  /?  =  0  then  /3[n]  is  empty  so 
its  cardinality  is  0  =  (?o(n).  If  ^  >  0  then  I3[n]  =  Pn(/?)H  U  {Pn(/?)}  and  so 

card^[n]  =  1  + card  Pn(/?)[7i]  =  H-Gp„(^)(n)  =  Gp{n) , 

Lemma  4.3 


=  H^{9)  o  Hp{g) 


and 


H20  =  . 
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Proof,  (i)  By  induction  on  /?.  The  first  identitiy  holds  automatically  if  ^  =  0 
since  Ho{g)  is  just  the  identity.  If  ^  >  0  then  by  the  earlier  definition  of  7  +  0[n] 
we  see  that  Pn(7  +  /3)  =  7  +  Pn{P)  and  so 

=  Hp^^^^i3){g){9{n)) 

=  H^+Pr.{0)(9)i9iri)) 

=  H^{g)  o  Hp^(0){g)  {g(n)) 

=  H^{g)  o  Hp{g)  (n)  . 

(ii)  Again  by  induction  on  13.  li  f3  —  0  then 

H2o{g){n)  =  Hi{g){n)  =  Ho(g){g{n))  =  g{n)  =  BQ{g){n). 

li  /3  >  0  then  by  inspecting  the  definition  of  2^[n]  given  earlier  we  see  that 
Pn{2^)  =  and  therefore  using  the  first  part  of  the  Lemma 

we  have 

H2^{9)M  “  Hp^(2PnW^2Pr^il3)){9){9W) 

=  H2PniP)+2f*n(P)(g){n) 

=  H2Pni^)  {g)oH2Pnw{9)  (n) 

=  Bp^(0){g)  0  Bp^(0){g)  (n) 

=  Bp(g){n)  by  definition  of  B. 

Remark.  As  we  shall  see,  the  above  relationship  between  the  doubly  nested 
definition  of  B  and  the  tail  recursive  definition  of  H  is  in  fact  a  cut  elimination 
result. 


5  Ordinal  Bounds  on  Equational  Derivations 

This  section  is  based  on  work  of  Fairtlough  (1991),  Fairtlough-Wainer  (1992) 
and  Handley- Wainer  (1994).  The  idea  is  to  assign  uniform  ordinal  bound?  to 
derivations  of  recursive  functions  in  Kleene’s  Equation  Calculus,  from  which  we 
can  read  off  sub-  recursive  hierarchy  classifications  of  their  computational  com¬ 
plexity.  (Complexity  is  viewed  “in  the  large”  -  we  are  not  concerned  here  with 
questions  about  feasible  or  polynomially  bounded  computation,  but  rather  with 
general  methods  for  comparing  the  complexities  of  natural  classes  of  recursive 
definitions.) 

Let  E  be  a  system  of  recursion  equations  (i.e.  a  recursive  program)  defining 
functions  /o,  •  •  ■ ,  /fe  from  “given”  functions  =  po,  •  •  ■ , P/  including  the  constant 
zero,  the  identity  and  the  successor.  Let  t  denote  a  variable-free  term  built  up 
from  numerals  by  repeated  applications  of  the  function  symbols  fi  and  gj.  Write 
to  -^E  h  if  to  rewrites  to  ti  in  one  step  by  replacing  any  subterm  /i(n)  by  its 
defining  term  in  E.  Let  ordinal  a  be  given  with  a  fixed  presentation. 

Then  the  relation 


n:N  }-%  t  =  m 
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meaning  “from  inputs  less  than  or  equal  to  n  we  can  derive  that  t  has  value 
m  within  ordinal  bound  a”,  is  generated  inductively  according  to  the  following 
rules  (we  omit  the  “E”  although  the  dependence  on  a  given  system  of  defining 
equations  is  to  be  understood)  : 

Ax  n  :  AT  f-“  g{n)  =  m  if  maxn  <  n  and  ^'(n)  has  value  m. 


E 


n:N  ti  =  m 
n:N  h"  t{i=m 


if  to  ~~^E  ti* 


g  n  :  AT  to  =  fc  max(n,  k)  :  N  ti{k)  =  m 
n  :  N  ti(to)  =  m 

R  In  the  E  and  C  rules  the  restriction  is  that  j5,/5o,/?i  G  a[n]. 


Remark.  Of  course  any  derivation  oiniN  t  =  m  is  finite,  so  why  need  we 
consider  possibly  infinite  ordinal  bounds  ?  The  reason  is  that  if  /  is  a  function 
defined  by  a  system  of  equations  E,  then  what  we  want  to  do  is  find  a  uniform 
bound  a  such  that 


Vn  G  AT  (/(n)  =  m  n  :  N  \-%  f(n)  =  m). 

It  is  the  recursion  rule  R  which  allows  us  to  do  this.  Then  a  will  provide  an 
abstract  measure  of  the  complexity  of  the  program  defining  /. 

Lemma  5.1  Suppose  n  :  N  t  =  m.  Let  g  now  denote  any  fixed  positive, 
strictly  increasing  unary  function  which  hounds  all  the  givens  go,....,gi  in  E, 
and  let  g*  =  ^  4- 1.  Then  m  <  jBa(p)(n)  and  the  height  of  the  derivation  tree  is 
bounded  by  Ba{g^){n). 

Proof.  By  induction  over  the  derivation  of  ti  :  iV  t  =  m  according  to 
the  Ax,  E,  and  C  rules  restricted  by  R.  The  Ax  and  E  rules  are  trivial  since  if 
max(n)  <  n  then  for  each  j  <l  we  have,  for  e  a[7i], 

9jin)  <  9{n)  <  R/?(^)(n)  <  Ba{9){n)  <  Ba{9'){n)> 

Now  suppose  n\N  t  =  m  arises  by  the  C  rule  applied  to  premises 

n  \  N  to  =  A;  and  max(n,/:)  :  N  ti{k)  =  m  . 

Then  by  the  induction  hypothesis,  and  since  /?o,  A  €  a[n], 

max{n,A;)  <  B^,(p)(n)  < 

and 

m  <  B0^{g)(max{n,k))  <  Bp^(a)(g)(max(n, k)) . 

Therefore 

"I  <  ■Bp„(a)(fl)(  Bp^(o)(ff)(n) )  =  Ba{g){n). 
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Furthermore  the  height  of  the  derivation  -  viz.  the  least  number  greater  than 
the  heights  of  its  two  premises  -  is  bounded  by 

max  {Bp,{g'){n),B0,{9^){max{n,k))}  4- 1 

<  max{j5^o(5')W>^j9i(p')(^Pn{a)(p)W)}  +  1 

<  max{Bp„(a)(pOW  +  1, J5p^(a)(5')(-Spn(a)(i^)(w)  +  1)} 

<  ^p„(a)(^0(  ^P„(a)(ff)W  +  1  ) 

<  5p^(a)(p')(  ^Pn{ot){9^){'^)  ) 

=  Ba{9'){n)  . 

Definition.  Let  a  be  an  infinite  countable  ordinal  with  a  fixed  chosen  presen¬ 
tation.  Then  there  is  a  bijection  tt  :  a  -►  iV  and  so  we  can  represent  each  /?  <  a 
as  a  number  b  =  n{/3)  and  each  predecessor  function  Pn  a.  number-theoretic 
function  pn  such  that  Pn(b)  =  7r(P„(/5)).  We  can  assume  without  loss  of  gener¬ 
ality  that  0  =  7r(0).  Then  we  say  that  a  function  /  is  defined  by  a  -  recursion 
from  the  givens  h  and  g  if  for  some  term  T 

/(0,n)  =  h{n) 

f{b,n)  =  T{g,f;n) 

wherein  each  recursive  call  on  /  inside  T  is  of  the  form  f{pto{b),ti)  for  certain 
subterms  to  and  ti. 

Lemma  5.2  Suppose  f  is  defined  from  the  givens  h,g  by  the  a  -  recursion  equa¬ 
tions  E  as  above.  Then  there  is  a  fixed  k  E  N  such  that  for  every  b  =■  7r{fi)  we 
have 

\fne  N  (f{b,n)  =  m  =>  n  :  N  /(6,n)  =  m). 

Proof,  Choose  k  to  be  the  length  (number  of  symbols)  of  the  term  T,  plus 
1.  Proceed  by  induction  on  <  a.  The  case  6  =  0  follows  from  the  axiom 
n  :  iV  h®  h{n)  =  m  by  one  application  of  the  E  rule,  since  /(O,  n)  -^e  h{n)  and 
0  €  k[n]  as  long  as  A;  >  0.  Now  assume  6  >  0  and,  by  the  induction  hypothesis, 
that  the  result  holds  for  each  recursive  call  /(Pmo(^)>^i)  ^sed  in  evaluating 
/(6,n)  =  T{g,f]n).  We  prove,  by  an  inner  induction  over  all  subterms  t  of 
T{g,  /;n),  that  if  t  has  length  i  and  value  m,  then 

n  :  N  t  =  m  * 

The  desired  result  then  follows  immediately  for  if  t  is  the  whole  term  T  we  have 

n  :  N  r  =  m 

and  hence  n  :  N  fib^n)  =  m  follows  by  one  further  rewriting  step. 

To  prove  *  consider  three  cases.  Either  i  is  n  in  which  case  *  is  just  an 
axiom.  Or  t  is  of  the  form  g{to, . . .  ,tr)  with  g  a  “given”,  in  which  case  *  follows 
straightforwardly  by  r  +  1  applications  of  the  C  rule.  Or  else  t  is  of  the  form 
f{pto{b)iti)  for  certain  subterms  to  and  ti  where 

n  :  N  to  =  mo  and  n  :  N  ti  =  mi  . 
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In  this  CcLse,  by  the  overall  induction  hypothesis  on  we  have  for  the  appropriate 
value  m, 

mi-.N  /(p„„(6),mi)  =  m 

and  since  k.{Pma{P)  +  C  A:.j8[max(n,mo,mi)],  we  can  also  derive 

max(n,mo,TOi) :  J\r  /(Pm„(6),mi)  =  m  . 

Therefore  by  two  successive  applications  of  the  C  rule  we  obtain  first, 
max(n,jno)  f{Pmo{b),h)  =  m 

and  then,  since  the  length  i  of  term  t  is  at  least  io  +  ii  +  1, 
n:N  /(p,o(5),ti)=m. 

This  completes  the  proof. 

Definitions.  Let  REC{<  r)  denote  the  class  of  all  functions  definable  from 
the  constant,  addition  and  (modified)  subtraction  functions,  by  repeated  appli¬ 
cations  of  composition  and  a  -  recursion  with  a  <  r. 

Let  DER{<  t)  denote  the  class  of  all  functions  /  for  which  there  is  a  recursive 
program  E  (with  constants,  addition  and  subtraction  as  the  only  givens)  and  an 
ordinal  a  <t  such  that 

Vn  €  (/(^)  =  m  =>  maxn  :  N  f{n)  =  m)  . 

Let  COMP{B<r)  denote  the  class  of  all  functions  computable  (in  any  rea¬ 
sonable  model  of  computation)  with  resource  bound  for  some  a  <  r.  Since 
Buj  is  exponential  the  size  of  B^,  even  for  relatively  small  infinite  a,  already 
blurs  any  fine  distinctions  between  choices  of  computational  models. 

Theorem  5.3  Suppose  r  is  a  limit  ordinal  closed  under  addition  and  >  u). 
Suppose  also  that  we  have  a  fixed  presentation  of  r  and  a  hijection  tt  :  r  —*■  N 
such  that  (i)  for  every  <t,  e  T[7r(/5)],  and  (ii)  the  function  (n,6)  Pn(^) 

is  primitive  recursive.  Then 

REC{<t)  =  \jDER{<k-T)  =  \JCOMPiB<k  r)  ■ 
k  k 

Proof,  (i)  To  show  REC{<  r)  C  Ujk  ^  •  t)  suppose  for  example  that  / 

is  defined  by  an  a  -  recursion  from  given  functions  g  where  a  <r  and  where  it 
is  inductively  assumed  that  g  €  DER{<  k  •  7)  for  sufficiently  large  k.  Then  by 
Lemma  5.2  there  is  an  appropriate  k  such  that  for  all  n  and  all  b  =  7r{fi)  with 
0<a, 

n-.N  /(6,n)=m. 

If  we  insert  above  the  axioms  introducing  the  givens  p,  their  derivations  with 
bound  A;  ■  7,  we  obtain 


n\N  /(6, n)  =  m 
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and  hence 

max(6,n)  :  N  —  m 

where  this  is  now  an  “outright”  derivation  from  axioms  defining  only  addition 
and  subtraction.  Now  since  /?  G  t[6]  fl  a  =  a[h]  we  have 

A:  •  (7  +  G  A;  •  (7  +  a)[b]  C  A;  •  (7  +  a)[max(&,n)] 


and  hence 


A;  •  (7  +  /?  +  l)[max(6,  n)]  C  fc  •  (7  +  Q!)[max(6,  n)]  . 

Therefore  we  obtain  for  all  6,  n  G  iV, 

max(6,  n) :  AT  /(6,  Ti)  =  m 

and  since  r  is  closed  under  addition,  7  +  a  <  r,  so  /  belongs  to  DER{<  k-r). 

(ii)  To  show  DER{<  k-r)  C  COMP{B<k  r)^  suppose  /  is  defined  by  a  recursive 
program  E  (with  only  addition  and  subtraction  as  givens)  in  such  a  way  that  for 
some  a  <  A;  •  r, 


Vfi(/(n)  =  m  maxn  :  N  f{n)  =  m)  . 

Then  by  Lemma  5.1,  the  finite  height  of  the  derivation  tree  for  /(n),  and  all  the 
values  m  of  subcomputations  performed  therein,  are  bounded  by  Ba(p)(maxn) 
for  some  linear  function  g.  So  for  each  input  n  the  entire  derivation  tree  for  f(n) 
can  be  encoded  as  a  machine  computation  with  resource  bound  exponential  in 
Ba(^).  Since  Ba,(n)  =  B^{succ){n)  =  n  +  we  have  g  <  Bk.^j  provided 
Jb  >  0,  and  hence  Ba{g)  <  Bk.u;-hot  and  hence  the  resource  bound  in  computing 
/  will  be  less  than  B^  o  Ba{g)  which  in  turn  is  less  than  Bk-u^^-a  ^  Bk.u}-\-a  = 
But  fc-a;  +  a-i-l<A;*T  and  hence  /  G  COMP(B<kr)' 

(iii)  To  show  COMP{B<k  r)  ^  REC{<  r)  we  only  need  to  prove  that  Bk-a  ^ 
REC{<  t)  for  every  a  <  r,  because  if  /  is  computable  within  resource  bound 
Bk-a  then  /  is  definable  from  Bk  a  compositions  and  u)  -  recursions  only,  and 
so  /  will  also  be  in  REC{<  r).  But  notice  that 

BkaM  =  Bk.p^{oc)+k{n)  =  5fc.p„(a)(w) 

the  2^  -  times  iterate  of  So  Bk-a  is  definable  by  a  2^  -  times  nested  a 

-  recursion,  and  therefore  it  belongs  to  REC{<  r). 

This  completes  the  proof. 

Examples.  REC{<  w.2)  is  the  class  of  all  Csillag-Kalmar  elementary  functions, 
REC{<  is  the  class  of  all  primitive  recursive  functions,  and  REC{<  €0)  is 
the  class  of  all  provably  recursive  functions  of  Peano  Arithmetic. 
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Derivations  of  Tail  Recursions.  A  tail  a  -  recursion  from  givens  h  and  g  is 
a  definition  of  the  form 


/(0,n)  =  h{n) 
fM  =  f{pn{b),9{n)). 

Following  the  idea  of  Lemma  5.2  we  notice  that  the  derivation  of  the  recursion 
step  uses  only  a  simple  form  of  the  C  rule  where  the  left  hand  premise  is  an 
axiom  : 


n:N  g{n)=m^  m' :  N  f{pn{b),m')=m 

_ n  :  N  f{pn{b)^g{n))  ~  m _ 

n  :  N  l-2-^  f{b,n)  =  m 

Thus  if  P  e  Q:[7r{/?)]  we  can  prove  inductively  that 

max(6,7i) :  N  I-q  “  f(b,n)  =  m 

where  ho  now  denotes  derivability  using  the  Ax  and  E  rules,  but  only  these  re¬ 
stricted  forms  of  the  C  rule. 

Definitions,  (i)  Let  rA/L(<  r)  denote  the  class  of  functions  definable  from 
constants,  addition  and  subtraction  by  repeated  compositions  and  tail  a  -  re¬ 
cursions  where  a  <  r. 

(ii)  Let  DERo{<  r)  denote  the  class  of  all  functions  /  for  which  there  is 
a  recursive  program  E  (with  constants,  addition  and  subtraction  as  the  only 
givens)  and  an  ordinal  a  <  r  such  that 

Vn  €  (f(n)  =  m  =>  maxn  :  N  hj  f{n)  =  m)  . 

Theorem  5.4  Let  r  satisfy  the  same  conditions  as  in  the  previous  Theorem. 
Then 


TAIL{<t)  =  Uj[)£i?o(<A:-T)  =  \JCOMP{H<kr)  ■ 

k  k 

Proof.  The  above  comments  are  enough  to  show  how  the  proof  of  the  first  part 
of  Theorem  5.3  can  be  modified  to  yield  TAIL{<  r)  C  DERo{<  ib  •  r),  in  fact 
with  k  =  2. 

The  proof  of  DERq{<  k-r)  C  COMP{H<:k  r)  follows  exactly  the  same  lines 
as  the  proof  of  the  second  part  of  Theorem  5.3.  But  instead  of  using  Lemma  5.1 
we  now  need  a  corresponding  lemma  giving  the  appropriate  bounding  functions 
for  ho  derivations.  Since  only  a  restricted  form  of  the  C  rule  is  allowed  here,  a 
careful  inspection  of  the  proof  of  5;1  shows  that  it  is  the  H  -  functions  which 
now  give  the  appropriate  bounds.  For  if  we  have  a  derivation 

n  :  AT  h^  g{n)  =  m'  m’  :  N  t(m‘)  —  m 
n  :  AT  Kg'  t(g{n))  -  m 
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where  inductively  we  assume  m  <  H0{g){m^)  then  since  ^  E  Oi[n]  we  obtain 

m  <  Hp{g){g{n))  <  i?p„(a)(p)(pW)  =  Hoc{g)in)  . 

The  proof  of  COMP{H<k’r)  ^  TAIL{<  r)  is  again  similar  to  the  final  part 
of  the  proof  of  5.3.  For  any  a  <  r  it  is  easy  to  see  that 

Hk.a{9){n)  =  Hk.p„^a)+kig){n)  =  ■H'*p„(a)(fl)(p*(«)) 

and  so  Hk-aig)  is  definable  by  a  tail  a  -  recursion  from  the  given  function  g*‘. 
This  completes  the  proof. 

Theorem  5.5  Let  r  satisfy  the  same  conditions  as  in  the  previous  Theorems. 
Then  for  each  k, 

DER{<k-r)  =  DERo{<2'‘"^) 

and  hence 

REC(<  r)  =  U  TAIL{<  2'=  '^) . 

k 

Proof.  Simply  recall  that  Bp  =  H213  and  apply  the  above  Theorems. 

Remark.  In  terms  of  equational  derivation,  this  is  a  cut  elimination  result. 
The  first  part  shows  that  the  “call  by  value”  cut  rule  C  can  be  reduced  to  the 
restricted  form 

n:  N  \-  f{n)  =  m 

at  the  cost  of  an  exponential  increase  in  the  ordinal  presentation.  And  the  second 
part  shows  that  this  cut  reduction  amounts  to  the  transformation  from  general 
recursive  to  tail  recursive  programs. 

This  prompts  a  further  question.  Since  tail-recursion  corresponds  to  the  cut 
rule  in  which  the  left  hand  premise  is  an  axiom,  what  kind  of  recursion  is  mod¬ 
elled  by  the  cut  rule  in  which  the  right  hand  premise  is  an  axiom  ? 

Derivations  of  Pointwise  Recursions.  A  pointwise  a  -  recursion  from  givens 
h  and  p  is  a  definition  of  the  form 

/(0,n)  =  h{n) 

f  ib,n)  =  g{f{pn(b),n)) . 

Notice  that  the  parameter  n  remains  fixed  in  the  recursive  call  on  /  and  so  the 
derivation  of  the  recursion  step  uses  only  a  C  rule  where  now  the  right  hand 
premise  is  an  axiom  : 

n  :  N  f{Pn{b),n)  =  k  k  :  N  g{k)  =  m 

n:N  p--Pn(/?)+i  g{f{pn{b),n))  =  m _ 

n  :  N  f(b^n)  =  m 
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Thus  if  /3  €  a  [6]  where  b  =  Tr{fi)  we  can  prove  inductively  that 
max(6,n)  :  N  “  /(6,n)  =  m 

where  |=  now  denotes  derivability  using  the  Ax  and  E  rules,  but  only  these  right- 
hand  restricted  forms  of  the  C  rule.  Since  the  ordinal  bounds  on  this  derivation 
of  f{b,  n)  all  now  lie  in  the  finite  set  2.Q![max(6,  n)]  of  cardinality  G2.a(niax(6,  n)), 
it  is  clear  that  the  complexity  of  |=  -derivations  will  be  bounded  by  the  Slow 
Growing  hierarchy.  Thus  for  appropriate  r ’s  satisfying  the  additional  condition 
that  for  each  pair  ao,  ai  <  r  there  is  an  a  <  r  such  that  G^o  o  Ga^  is  bounded 
by  Gai  we  have 


POINTWISE{<t)  =  COMP{G<r)‘ 

Comparisons  between  the  G  and  B  hierarchies,  and  hence  between  pointwise 
recursion  and  arbitrary  nested  recursion,  are  more  subtle  than  the  exponential 
trade-off  between  B  and  H.  However  from  Girard  (1981)  and  others  subsequently 
(seq  Wainer  (1989)  for  direct  computations)  we  have  a  map  a  a***  from  ordinal 
presentations  to  ordinal  presentations,  defined  at  least  on  the  initial  segment 
below  the  proof  theoretic  ordinal  of  (Hj  -  CA)o  ,  such  that 

=  Ga+  . 

Consequently  for  appropriate  r ’s  we  obtain 

REC{<t)  =  POINTWISE{<t-^)  . 

Examples.  For  the  primitive  recursive  functions  we  have  t  =  and  =  (a 
presentation  of)  the  first  primitive  recursively  closed  ordinal.  For  the  provably 
recursive  functions  of  arithmetic  we  have  r  =  eo  and  —  the  Howard  ordinal. 
See  e.g.  Schwichtenberg- Wainer  (1995). 

The  significance  of  pointwise  recursion  is  that  it  is  intimately  connected  with 
term-rewriting.  The  work  of  Cichon  (1992),  Weiermann  (1991,1993,1995)  and 
Handley- Wainer  (1994)  shows  how  the  ordinal  gives  a  “termination  ordering” 
along  which  the  <  r  -  recursive  functions  are  computable  by  rewriting.  Thus 
the  -h  -  map  and  the  associated  cut  elimination  from  h  to  |=  reflect  a  trade-off 
between  derivations  of  “call-by- value”  recursion  on  one  hand,  and  computability 
by  rewriting  on  the  other. 
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Abstract.  We  define  a  notion  of  a  Scott  family  of  formulas  for  a  feasible 
model  and  give  various  conditions  on  a  Scott  family  which  imply  that 
two  models  with  the  same  family  are  feasibly  isomorphic.  For  example,  if 
A  and  B  possess  a  common  strongly  p-time  Scott  family  and  both  have 
universe  {!}*,  then  they  are  p-time  isomorphic.  These  results  are  applied 
to  the  study  of  permutation  structures,  linear  orderings,  equivalence  re¬ 
lations,  and  Abelian  groups.  For  example,  conditions  on  two  permutation 
structures  (A,/)  and  {B,g)  are  given  which  imply  that  {A,  f)  and  (B,p) 
axe  p-time  isomorphic. 


Introduction 

The  focus  of  feasible  mathematics  has  been  on  determining  the  complexity  of  cer¬ 
tain  classes  of  models.  The  classic  example  is  the  graph-coloring  problem,  where 
it  is  known  that  the  family  of  finite  graphs  which  can  be  3-colored  is  a  complete 
NP  class.  The  problem  of  feasible  colorings  of  infinite  graphs  was  studied  by  the 
authors  in  [4].  Polynomial  time  model  theory  or  more  generally  feasible  model 
theory,  the  subject  of  this  paper,  considers  the  complexity  of  the  model  itself. 
Let  cj  =  {0, 1, . . .}  denote  the  set  of  natural  numbers.  Let  (, )  denote  some  fixed 
recursive  pairing  function  which  maps  lj  x  onto  u.  Let  (j>e,n  denote  the  n-ary 
partial  function  on  ({0,1}*)”  computed  by  the  e-th  Turing  machine.  Then  we 
say  that  a  structure  A  =  (A,  {fi^}i£T,  {c^}ieu)i  (where  the  universe 

A  of  A  is  a  subset  of  (0, 1}*)  is  recursive  if  A  is  a  recursive  subset  of  {0, 1}*, 
5,T,  and  U  are  initial  segments  of  a;,  the  set  of  relations  {Rf}ies  is  uniformly 
recursive  in  the  sense  that  there  is  a  recursive  function  G  such  that  for  alH  e  5, 
G{i)  —  {ni,ei)  where  Rf  is  an  rij-ary  relation  and  0ei,ni  computes  the  charater- 
istic  function  of  the  set  of  functions  {f^}iqT  is  uniformly  recursive  in  the 
sense  that  there  is  a  recursive  function  F  such  that  for  alH  G  T,  F{i)  —  (ni,ei) 
where  ff-  is  an  n^-ary  function  and  <l>ei,ni  restricted  to  A”  computes  and 
there  is  a  recursive  function  interpreting  the  constant  symbols  in  the  sense  that 
there  is  a  recursive  function  H  such  that  for  all  i  G  C/,  H{i)  —  cf.  Note  that  if  A 
is  a  recursive  structure,  then  the  atomic  diagram  of  A  is  recursive.  We  say  that 
a  recursive  structure  A  =  (A,  {fi^}ieT,  {cf}ieu)y  is  polynomial  time 

if  A  is  a  polynomial  time  subset  of  {0, 1}*  and  the  set  of  relations  {Rf}i^s  and 
the  set  of  functions  {fj^}ieT  are  uniformly  polynomial  time  in  the  sense  that,  in 
addition  of  the  functions  G  and  F  defined  above,  there  are  recursive  functions  G' 
and  F'  such  that  for  i  G  5,  G'{i)  =  m*  where  for  all  in  ({0, 1}*)”*, 

it  takes  at  most  (maa:{2,  |a;i|, . .  • ,  steps  to  compute  (l>ei,ni{xij . . .  jaJnJ 
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and  for  all  i  G  T,  G'{i)  =  qi  where  for  all  (xi , . . . ,  )  in  ({0,  !}*)”• ,  it  takes  at 
most  (maa;{2,  |a:i|, . . . ,  |a;„.  1})*^*  steps  to  compute  (j)ei,ni  (a:i , . . . ,  )•  Note  if  A  is 

polynomial  time  structure  with  infinitely  many  relation  symbols  or  with  infinitely 
many  function  symbols,  then  our  definition  of  a  polynomial  time  structure  does 
not  ensure  that  the  atomic  diagram  of  A  is  polynomial  time.  Thus  we  say  A  is 
uniformly  polynomial  time  if  the  atomic  diagram  of  A  is  polynomial  time.  Note 
that  the  fact  that  A  is  uniformly  polynomial  time  implies,  among  other  things, 
that  the  sequence  of  run  times  :  i  €  S}  and  {x^'  :  i  eT}  are  bounded  by 
some  fixed  polynomial.  Of  course,  if  ^4  is  a  structure  over  a  finite  language,  then 
^  is  a  polynomial  time  structure  iff  A  is  uniformly  polynomial  time  structure. 

There  are  two  basic  types  of  questions  which  have  been  studied  in  polynomial 
time  model  theory.  First,  there  is  the  basic  existence  problem,  i.e.  is  a  given  infi¬ 
nite  recursive  structure  A  isomorphic  or  recursively  isomorphic  to  a  polynomial 
time  model.  For  example,  the  authors  showed  in  [1]  that  every  recursive  rela¬ 
tional  structure  is  recursively  isomorphic  to  a  polynomial  time  model  and  that 
the  standard  model  of  arithmetic  (a;, 2®)  with  addition,  subtraction, 
multiplication,  order  and  the  1-place  exponential  function  is  isomorphic  to  a 
polynomial  time  model.  A  more  restricted  kind  of  existence  question  is  whether 
a  given  recursive  model  is  isomorphic  or  recursively  isomorphic  to  a  polynomial 
time  model  which  has  a  standard  universe  such  as  the  binary  representation  of 
the  natural  numbers,  Bin{uj),  or  the  tally  representation  of  the  natural  numbers, 
Tal{uj)  =  {1”  :  71  G  a;}.  For  example,  Grigorieff  [8]  proved  that  every  recursive 
linear  ordering  is  isomorphic  to  a  linear  time  linear  ordering  which  has  universe 
Bin{Lj)  while  Cenzer  and  Remmel  [1]  showed  that  there  exists  a  recursive  copy 
of  the  linear  ordering  w  -f  cj*  which  is  not  recursively  isomorphic  to  any  polyno¬ 
mial  time  linear  ordering  which  has  universe  Bin{uj).  Here  w-f  cj*  is  the  ordering 
obtained  by  taking  a  copy  ofa;  =  {0,l,2,...}  under  the  usual  ordering  followed 
by  a  copy  the  negative  integers  under  the  usual  ordering.  The  general  problem  of 
determining  which  recursive  models  are  isomorphic  or  recursively  isomorphic  to 
feasible  models  has  been  studied  by  the  authors  in  [1],  [2],  and  [5].  For  example, 
it  was  shown  in  [2]  that  any  recursive  torsion  Abelian  group  G  is  isomorphic 
to  a  polynomial  time  group  A  and  that  if  the  orders  of  the  elements  of  G  are 
bounded,  then  A  may  be  taken  to  have  a  standard  domain,  i.e.  either  Bin{uj)  or 
Tal{u)),  Feasible  linear  orderings  were  studied  by  Grigorieff  [8],  by  Cenzer  and 
Remmel  [1],  and  by  Remmel  [14]  [16].  Feasible  vector  spaces  were  studied  by 
Nerode  and  Remmel  in  [10]  and  [12].  Feasible  Boolean  algebras  were  studied  by 
Cenzer  and  Remmel  in  [1]  and  by  Nerode  and  Remmel  in  [11].  Feasible  permuta¬ 
tion  structures  and  feasible  Abelian  groups  were  studied  by  Cenzer  and  Remmel 
in  [2]  and  [5]. 

The  second  basic  type  of  problem  studied  in  polynomial  time  model  theory 
is  the  problem  of  feasible  categoricity.  Here  we  say  that  a  recursive  model  A  is 
recursively  categorical  if  any  other  recursive  model  isomorphic  to  A  is  in  fact 
recursively  isomorphic  to  A.  Defining  a  natural  analogue  of  feasible  categoricity 
is  complicated  by  the  fact  that  unlike  the  case  of  infinite  recursive  models,  where 
any  two  infinite  recursive  universes  are  recursively  isomorphic,  it  is  not  the  case 
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that  any  two  polynomial  time  universes  are  polynomial  time  isomorphic.  For 
example,  Bin{Lj)  is  not  polynomial  time  isomorphic  to  Tal{uj).  It  turns  out  to 
be  more  natural  to  define  polynomial  categorical  structures  with  respect  to  a 
fixed  universe.  Thus  we  say  that  a  p-time  structure  A  with  universe  D  C  {0, 1}* 
is  p-time  categorical  with  respect  to  D  if  every  p-time  structure  B  with  universe 
D  which  is  isomorphic  to  A  is  necessarily  p-time  isomorphic  to  A^  i.e.  there  exist 
polynomial  time  functions  f,g  such  that  /  restricted  to  D  is  an  isomorphism 
from  A  onto  B  and  g  restricted  to  D  is  an  isomorphism  from  B  onto  A.  For 
example,  the  problem  of  feasible  categoricity  for  permutation  structures  and 
torsion  Abelian  groups  was  studied  by  Cenzer  and  Remmel  in  [5].  A  permutation 
structure  is  a  structure  (A,  /)  where  •/  :  A  A  is  a  permutation  of  A.  The 
following  results  on  poynomial  time  categoricity,  and  other  notions  of  computable 
categoricity,  of  permutation  structures  were  proved  in  [5]. 

(1)  A  recursive  permutation  structure  is  recursively  categorical  if  and  only  if  it 
has  only  finitely  many  infinite  orbits. 

(2)  If  the  recursive  permutation  structure  A  is  not  recursively  categorical,  then 
there  exist  p-time  structures  Bi  and  each  isomorphic  to  A,  having  tfie 
same  standard  universe  (either  Bin(u;)  or  Tal{u))),  which  are  not  recursively 
isomorphic  to  each  other. 

(3)  If  the  recursive  permutation  structure  A  has  an  infinite  orbit,  or  has  infinitely 
many  orbits  of  size  for  some  finite  g,  and  infinitely  many  other  orbits,  then 
there  exist  p-time  structures  Bi  and  B2^  each  isomorphic  to  A,  having  the 
same  standard  universe  (either  Bin{(jo)  or  Tal{ijj)),  which  are  not  primitive 
recursively  isomorphic  to  each  other. 

(4)  If  A  =  (A,  /)  and  B  =  (B,  g)  are  two  p-time  permutation  structures  having 
only  finite  orbits  and  all  but  finitely  many  orbits  have  the  same  finite  size 
q,  then  A  and  B  are  p-time  isomorphic  if  B  =  Tal{ijj)  and  are  exponential 
time  isomorphic  if  B  =  Bin{(jj). 

We  note  that  Remmel  [14]  and  Dzgoev  [7]  proved  that  a  recursive  linear  ordering 
L  is  recursively  categorical  iff  L  has  finitely  many  successivities,  i.e.  iff  L  has 
only  finitely  many  pairs  x  <l  y  where  y  is  an  immediate  successor  of  a:  in  L. 
Remmel  showed  in  [16]  that  there  is  in  fact  no  p-time  categoricity  for  linear 
orderings.  That  is,  for  any  p-time  linear  ordering  L  ~  (B,  <)  with  standard 
universe  B  =  Bin{uj)  oi  B  =  Tal{uj),  Remmel  proved  the  following. 

(5)  If  L  has  infinitely  many  successivities,  then  there  exists  a  p-time  ordering  V 
with  universe  B  which  is  isomorphic  to  L  but  not  recursively  isomorphic  to 
L. 

(6)  If  1/  has  only  finitely  many  successivities,  then  there  exists  a  p-time  ordering 
V  with  universe  B  which  is  isomorphic  to  L  but  not  primitive  recursively 
isomorphic  to  L, 

These  results  are  typical  in  the  sense  that  they  show  that  polynomial  time  cat¬ 
egorical  structures  over  a  standard  universe  are  relatively  rare  and  that  polyno¬ 
mial  time  categoricity  can  fail  in  quite  strong  ways.  The  existence  questions  and 
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categoricity  questions  discussed  above  for  polynomial  time  models  can  easily  be 
generalized  to  other  notions  of  feasible  models  such  as  exponential  time,  poly¬ 
nomial  or  exponential  space,  etc.  The  goal  of  this  paper  is  to  develop  general 
conditions  under  which  some  form  of  feasible  categoricity  can  be  demonstrated. 
The  reader  is  refered  to  [9]  for  basic  notions  of  complexity  theory  and  to  [5]  or 
[16]  for  basic  notions  of  feasible  model  theory. 

1  Feasibly  categorical  structures 

The  specific  purpose  of  this  paper  is  to  develop  syntactic  approximations  of 
the  notion  of  feasible  categoricity.  Nurtazin  [13]  and  Goncharov  [6]  provided 
sufficient  conditions  to  ensure  that  a  model  A  with  universe  A  is  recursively 
categorical,  namely  if  there  is  a  finite  sequence  (co, . .  .,Ck-i)  of  elements  of  A 
and  a  recursive  sequence  (called  a  Scott  family  )  of  recursive  existential  formulas 

:  n  <  a;}  in  the  extended  language  with  names 
for  .Co, ... ,  Ck-i  satisfying  the  following  two  conditions: 

(1)  Every  ao?  •  •  •  >  cLm-i  €  A  satisfies  one  of  the  formulas  <l>n] 

(2)  For  each  n  and  for  any  two  sequences  (ao, . . . , am-i)  and  (do,  •  •  • , if 

A  satisfies  </>n(ao,  •  •  • ,  ®m— i, •  •  • , ^k—i)  and  </>n(^^o,  •  •  • , co, . . . , c/j—i), 

then  (yl,  tto, . . . , Co, . . . ,  Ck-i)  is  isomorphic  to 

(A,  do,  •  •  • ,  dm-i,  Co, . . . , Cjb_i)  via  the  map  which  sends  ai  to  di  for  i  <  m 

and  Ci  to  ci  for  i  <  k. 

There  are  three  observations  about  polynomial  time  (p-time)  models  which 
affect  the  notion  of  categoricity.  First,  there  are  p-time  subsets  of  {0, 1}*  which 
are  not  p-time  isomorphic,  for  example  Bin{uj)  and  Tal{u).  Thus  even  the  un¬ 
adorned  model  A  with  no  relations  or  functions  has  p-time  models  which  are 
not  p-time  isomorphic.  In  fact,  for  any  p-time  relational  structure  A,  there  is  a 
p-time  structure  B  which  is  recursively  isomorphic,  but  not  even  primitive  recur¬ 
sively  isomorphic  to  A,  see  [5] .  Thus  we  consider  categoricity  with  respect  to  a 
fixed  universe,  such  as  Tal{uj)  or  Bin{u)).  Second,  the  iteration  of  a  polynomial 
time  algorithm  does  not.  produce  a  p-time  function.  For  example,  if  /(O)  =  2 
and  f{n  -f  1)  =  f{ny,  then  /(n)  =  2^^.  Therefore,  we  often  need  to  relax  the 
complexity  of  the  isomorphism  between  p-time  structures.  For  example,  if  A  and 
B  are  two  p-time  groups  both  isomorphic  to  the  direct  sum  of  infintely  many 
cyclic  groups  of  order  p,  then  A  and  B  are  EXP  TIME  isomorphic  if  both  have 
universe  Tal{uj)  and  are  double-exponential  time  isomorphic  if  both  have  uni¬ 
verse  Bin{uj)y  but  are  not  necessarily  p-time  isomorphic  in  either  case,  see  [5]. 
Third,  we  must  consider  the  notion  of  honest  witnesses.  In  a  p-time  model,  the 
existence  of  an  element  a  such  that  <^(tt,  rro, . . .  does  not  guarantee  that 

we  can  compute  such  an  element  even  in  primitive  recursive  time.  Thus,  for  ex¬ 
ample,  there  are  two  p-time  models,  both  having  universe  Tal{(jj)j  of  the  simple 
structure  A  =  (A,i2),  where  R  is  an  infinite,  co-infinite  subset  of  A,  which  are 
not  even  primitive  recursively  isomorphic,  see  [5].  These  considerations  lead  us 
to  define  various  feasible  analogues  of  a  Scott  family.  For  example,  a  Scott  family 
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{(t>n{xi , . . . ,  Xjn-1 ,  Co,  Cl , . . , ,  Cfe_i )  :  71  <  cj},  foi  a  p-time  model  A  with  universe 
A,  satisfying  (1)  and  (2)  as  described  above  is  said  to  be  strongly  p-time  if  there 
is  some  fixed  integer  r  >  1  such  that  the  following  conditions  are  satisfied,  for 
each  m  >  0. 

(3)  For  any  finite  sequence  ao,...,am-i  of  elements  of  A,  we  can  compute  in 
time  <  (7naa;{2,77i,  |ao|,...,|a^_i|})^  a  formula  (j)t  from  the  list  such  that 

. . . ,  Co, Cl, ... , Cjb_i)  holds  in  A. 

(4)  For  each  formula  (aro , .  • . ,  ,  Co ,  Ci , . . . ,  cj^-i )  and  each  ao , . . . ,  am-\ 

€  Ay  if  there  exists  a  such  that  A  satisfies  0t(ao, . . . ,  am-i,  a,  cq,  ci, . . . ,  c^-i), 
then  there  exists  such  an  a  with  |a|  <  (m  +  2)^  +  maa:{|ao|, . . . ,  |arn-i|}- 

(5)  For  each  (f>t{xo, . . .  ,a^m-i,a;m,co,ci, . . .  ,ca:-i)  and  each  ao, . . .  ,am-i  €  A,  if 
there  exists  a  such  that  A  satisfies  (j>t{(io, .  • . , am-ij  co,  ci, . . . , c^-i),  then 
we  can  compute  an  a  as  described  in  (4)  in  time 

<  (7narr{2,m,  |ao|, . . . ,  |a^-i|})^. 

Theorem  1.  If  A  and  B  possess  a  common  strongly  p-time  Scott  family,  then  A 
and  B  are  p-time  isomorphic  if  both  have  universe  Tal{u})  and  are  exponential 
time  isomorphic  if  both  have  universe  Bin{(jS). 

Proof.  For  notational  convenience,  we  shall  give  the  proof  when  the  Scott  family 
has  no  parameters  as  the  proof  for  Scott  families  with  parameters  is  essentially 
the  same.  Also  for  any  a  G  cj,  we  shall  abuse  notation  and  simply  write  a  for  the 
tally  representation  of  a  when  the  universe  is  Tal(jjj)  and  write  a  for  the  binary 
representation  of  a  when  the  universe  is  Bin(uj).  The  isomorphism  0  between 
A  and  B  is  given  by  the  usual  back-and-forth  construction.  At  stage  n,  we  will 
have  defined  finite  sequences  ao,...,a„_i  and  such  that  the  map 

taking  Uj  to  bi  is  an  isomorphism  from  the  restriction  of  A  to  {ao, . . .  ,an-i} 
to  the  restriction  of  B  to  {bo,.. . ,6„_i}.  We  begin  at  stage  1  with  the  following 
procedure.  Let  ao  =  0.  Then  compute  the  formula  (l>t  such  that  A  satisfies  (^t(ao). 
Next,  we  compute  b  €  B  such  that  B  satisfies  (l>t{h)  and  we  let  bo  =  b.  At  stage  2, 

there  are  two  choices.  If  bo  =  0,  then  6i  =  1  and  if  bo  >  0,  then  bi  =  0.  Then  we 

compute  the  formula  <^t  such  that  <l>t{bo,bi)  holds  in  B.  Next,  we  compute  a  e  A 
such  that  <^t(ao,a)  holds  in  A  and  we  let  ai  =  a.  In  general,  at  stage  2m  +  1, 
we  let  a2m  be  the  least  a  not  equal  to  a^  for  any  i  <  2m.  Then  we  compute 
the  formula  <j)t  such  that  <l>t{ao,...  ,a2m)  holds  in  A.  Next,  we  compute  b  =  62m 
such  that  (f>t(bo,bi,. . . ,  b2m-i,b)  holds  in  B.  At  stage  2m  +  2,  we  similarly  let 
b2m+i  be  the  least  6  not  equal  to  bi  for  any  i  <  2m  + 1,  we  compute  the  formula 
(f>f  such  that  (/>t(bo,..., &2m+i)  holds  in  B  and  we  then  compute  a  =  a2m+i  such 
that  <j>t{ao, . .  .,a2mja,)  holds  in  A.  It  is  easy  to  see  by  induction  that,  for  each 
m, 

a2m  <  "^rn,  b2m+i  <  2m  +  1  (1) 

|a2m+i|  <  1  +  3^  -f  5^  +  •  •  •  +  (2m  +  3)^  <  (2m  +  5)^+\  (2) 

and  |62m|  <  2’’  +  4’'  +  . . .  +  (2m  -f  2)^  <  (2m  +  4)^+'.  (3) 

We  can  now  find  an  upper  bound  for  the  time  required  for  each  stage  of  the 
computation.  At  stage  2m  + 1,  given  ao, . . . ,  a2m-i  and  bo,...,  62m-i,  we  require 
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time  <  cm  to  compute  a2mi  for  some  constant  c.  Next,  we  require  time  < 
(2m  +  to  find  the  required  formula  <j>t.  Then  we  require  time  <  (2m  + 

2)^  +  (2m  +  2)^+^  to  compute  62m-  At  stage  2m  +  2,  we  require  time  cm  to 
compute  62m+ij  then  time  <  (2m  +  to  find  the  formula  <j>ty  and  finally 

time  <  (2m +  3)^  +  (2m +  4)^+^  to  compute  a2m+i-  Thus  the  total  time  required 
for  each  stage  n  is  bounded  by  kn^  for  some  fixed  constant  k.  It  follows  that 
the  total  time  required  for  stages  l,2,...,m  +  l  is  bounded  by  dm''  for 
some  fixed  constant  d.  Then  in  order  to  compute  0(a)  =  6,  we  must  follow  the 
construction  until  we  find  a  =  and  then  let  b  =  bm^  Now  we  know  from  the 
construction  that  a  =  am  for  some  m  <  2a.  Thus,  in  tally,  we  need  to  compute 
no  more  than  2a  stages,  so  that  we  can  compute  0(a)  in  time  <  d(2a)^  +r+i 
Since  in  tally  |a|  =  a  (for  a  >  0),  it  follows  that  0  is  polynomial  time.  A  similar 
argument  shows  that  0“^  is  also  p-time.  In  binary,  we  only  have  a  <  2l“l,  so 
that  the  first  2a  stages  of  the  construction  require  time 

d(2a)’'’+’'+^  (4) 

It  follows  that  0  is  exponential  time  and  similarly  that  0“^  is  also  exponential 
time.  □ 

We  apply  this  theorem  to  various  models  to  obtain  the  following  results. 

Corollary  2.  Let  A  =  (Ta/(t<;),/)  and  B  =  {Tal{uj)jg)  be  two  isomorphic  p- 
time  permutation  structures  such  that  for  some  fixed  integer  k, 

(i)  for  any  a  and  a'  in  the  same  orbit, 

W\  <  |a|  +  /:  (5) 

and 

(a)  for  any  ao,ai, . . .  ,am-i  6  B  and  any  finite  q,  if  there  is  an  orbit  of  size 
q  not  containing  any  of  Oq,  . . .  ,am-i,  then  there  is  such  an  orbit  containing  an 
element  a  of  size 

|al  <maa:{|oo|,...,|am_i|}  +  (m  +  2)*.  (6) 

Then  A  and  B  are  p-time  isomorphic. 

Proof.  First  observe  that  the  sizes  of  the  orbits  are  bounded  by  k.  The  Scott 
family  of  formulas  for  these  models  may  be  described  as  follows.  For  each  pair 
{x,y)  of  variables,  there  are  2k  basic  formulas,  P{x)  =  y  and  -^p{x)  =  y  for 
i  =  0, . . . ,  fc  -  1.  Then  for  any  finite  sequence  xq,  . . . , x^-i  of  variables,  there 
are  2fcm^  basic  formulas  ±f'{xh)  —  Xj.  k  formula  is  in  the  Scott  family  if  it 
is  the  conjunction  of  such  2km?'  basic  formulas  A/i, i,j  where  each  is 
i/*(^/i)  =  Given  a  finite  sequence  ao, . . . ,  a^-i  of  elements  of  A,  we  obtain 
the  formula  0(xo,  -  - . ,  Xm-i)  satisfied  by  ao, . . . ,  a^_i  by  computing  p{ah)  for 
a\\i  <k  and  comparing  the  result  to  each  aj.  Let 


n  =  max{2,m,ao, . . .  ,aT„_i} 


(7) 
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and  suppose  that  f{x)  can  be  computed  in  time  <  for  some  c.  It  follows 
that,  for  each  i  and  j,  \P{aj)\  <  \aj\^*  and  that  P{aj)  may  be  computed  in  time 

t  <  ‘  ^  •  (8) 

Thus  we  may  compute  a  list  of  all  the  /*(aj)  in  total  time 

t  <  kmn^^  <  (9) 

For  each  of  the  km^  pairs  of  bcisic  formulas  ±/*(x/i)  =  Xj,  we  then  compare 
f^{ah)  with  ttj  to  determine  which  formula  is  true.  This  can  be  done  in  time 
<  dn  for  sufficiently  large  d,  so  that  all  of  the  comparisons  can  be  done  in 
total  time  <  dkm?  <  The  formula  0  =  ^(xo, ... ,0:^-1)  satisfied  by 

ao, . . .  ,am-i  is  simply  the  conjunction  of  these  basic  formulas.  Thus  (j)  can  be 
computed  in  time 

t  <n^  where  r  =  dk  +  2k  +  4:,  (10) 

so  that  clause  (3)  in  the  definition  of  a  strongly  p-time  Scott  family  is  satisfied. 
Now  suppose  that  (/>(ao, . . .  ,ay„-i,a)  is  true  for  some  a,  where  (j)  is  one  of  the 
formulas  in  the  Scott  family.  There  are  two  cases.  First,  we  may  have  a  =  f^{aj) 
for  some  j.  Thus,  by  (i), 

H<|oj|  +  fc.  (11) 

Second,  we  may  have  a  in  some  orbit  of  size  q  disjoint  from  the  orbits  of 
flO  1  •  •  •  1  1  •  Then  by  (ii),  there  is  some  b  with 

|6|  <  mox{|oo|, . . . ,  |am_i|}  +  (m  +  2)*  (12) 

with  a  similar  property.  It  follows  from  the  definition  of  the  Scott  family  that 
(t){ao, . . .  6),  Thus  clause  (4)  is  satisfied.  The  computation  of  the  element 

a  is  likewise  in  two  cases.  If  a  =  P{aj)  for  some  i  and  j,  then  this  computation 
can  be  done  in  time  <  \aj\^*  as  indicated  above.  If  a  ^  for  any  i  anci  j, 

then  a  must  belong  to  a  new  orbit  and  we  can  read  the  size  q  of  that  orbit  from 
(j).  Then  we  search  for  the  least  6  with 


|6|  <  maa:{|ao|, . . . ,  +  (m  +  2)*  (13) 

which  is  different  from  every  P{aj)  and  such  that  f^{h)  =  b  and  p{b)  ^  b  for  any 
i  <  q.  We  may  assume  by  the  argument  above  that  we  have  already  computed 
the  orbits  of  each  of  the  elements  ao, . . . ,  am-i-  Then  the  choices  for  b  are  limited 
to 

6  <  n  +  (m  +  2)*  <  71 4-  (27i)*  <  71^*=+!  (14) 

and  each  choice  has  length  <  71^^+^.  Also,  there  are  <  mk  values  P{aj),  each 
of  length  <  71  +  A;  to  compare  b  with.  Then  for  each  b  and  each  ij  <  k,  the 
comparison  of  b  with  P{aj)  takes  time 

t<{n  +  kp  <  71*=+^ 


(15) 
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Since  there  are  (mfc)^  such  tasks,  the  total  time  required  for  this  comparison  is 

(16) 

It  is  also  necessary  to  compute  /‘(h)  fovi<q<k  and  b  <  and  to  check 

that  P{b)  ^  b  for  i  <  q  and  that  /®(h)  =  h.  Each  of  these  computations  take 
at  most  =  nc*(2*:+i)  steps  and  the  comparisons  take  time  at  most 

n{4*=+2)c«  ^  jg  easily  verified  that  clause  (5)  is  satisfied.  □ 

We  note  that  this  result  may  be  applied  to  permutation  structures  with  finitely 
many  finite  orbit  sizes.  For  example,  we  can  construct  a  model  {Tal{uj),  f)  satis¬ 
fying  the  hypothesis  of  Corollary  2  which  has  infinitely  many  orbits  of  size  2  and 
infinitely  many  orbits  of  size  3.  (Just  alternate  the  orbit  sizes,  so  that  f{x)  ~  x~l 
if  j:  =  l(mod  5),  f{x)  =  a;  -  2  if  x  =  4(mod  5)  and  f{x)  =  x  -1- 1  otherwise.) 
The  restrictions  (i)  and  (ii)  are  needed,  since  it  is  shown  in  [5]  that  there  are 
p-time  models  with  universe  Tal(uj)  isomorphic  to  but  not  primitive 

recursively  isomorphic  to  (Tal{u;),f).  On  the  other  hand,  if  all  but  finitely  many 
orbits  have  the  same  finite  size  g,  then  it  is  shown  that  even  without  conditions 

(i)  and  (ii),  any  p-time  model  with  universe  Tal{u)  is  actually  p-time  isomor¬ 
phic  to  A.  Our  next  result  was  proved  directly  by  the  second  author  in  [16].  To 
state  that  result,  we  need  to  define  when  a  polynomial  time  dense  linear  order 
without  endpoints  possesses  quasi-real  time  density  functions.  A  p-time  linear 
order  A  =  {A,  <^)  is  said  to  have  quasi-real  time  density  functions  if  there  exist 
functions  /,  p,  and  h  and  a  constant  c  such  that  for  all  a,  6  €  A  with  a  <"4  b: 

(i)  /(a)  may  be  computed  in  time  <  |a|  -h  c  and  /(a)  a, 

(ii)  p(a)  may  be  computed  in  time  <  |a|  +  c  and  a  9(0)1  and 

(iii)  h(a,  b)  may  be  computed  in  time  <  77iax{|a|,  \b\}  -h  c  and  a  h(a,  b)  b. 

If  the  run  time  bounds  in  clauses  (i),  (ii),  (iii)  are  replaced  respectively  by  < 
c  ■  |a|,  <  c  *  |al,  and  <  c  •  max{|al,  |6|},  then  we  say  that  /,  p,  and  h  are  linear 
time  density  functions  of  for  A.  Similarly  if  the  run  time  bounds  in  clauses  (i), 
(ii),  (iii)  are  replaced  respectively  by  <  (max{2,  |a|})^,  <  (max{2,  lal})^^,  and 
<  (max{2,  |a|,  \b\}y,  then  we  say  that  /,  p,  and  h  are  polynomial  time  density 
functions  of  for  A. 

Corollary  3.  Let  A  =  (Bm(a;),<^)  and  B  =  (J5m(a;),<^)  he  two  dense  p- 
time  linear  orderings  without  end  point  and  possessing  quasi-real  time  density 
functions.  Then  A  and  B  are  exponential  time  isomorphic. 

Proof.  We  will  outline  the  argument.  The  formulas  in  the  Scott  family  are  con¬ 
junctions  of  the  basic  formulas  Xi  <  Xj,  Xi  =  Xj.  Each  formula  gives  a  complete 
description  of  the  ordering  (with  equalities)  of  the  variables  involved.  We  com¬ 
pute  the  formula  satisfied  by  a  sequence  oq,  . . . ,  am-i  of  elements  of  A,  simply 
by  comparing  each  pair  of  elements.  It  is  easy  to  see  that  clause  (3)  is  satisfied. 
Any  formula  (/>t(xo, . . . ,  Xm-i  1  Xm)  describes  the  ordering  on  xq,  . . . ,  Xm-hXmi 
so  that  it  must  say  one  of  four  things  about  the  position  of  x„i.  Either  Xm  =  Xi 
for  some  i,  x^  <  Xi  where  Xi  <  xj  for.  all  j  <  m,  Xm  >  Xi  where  Xi  >  xj  for 


all  j  <  m,  or  xi  <  Xm  <  Xj  for  some  pair  i^j  such  that  Xj  is  the  successor  of  Xi 
in  the  ordering  on  {o^o, . . .  The  quasi-real  time  density  functions  /,  g,  h 

clearly  suffice  to  demonstrate  clause  (4)  as  well  as  clause  (5).  □ 

The  conditions  given  in  Theorem  1  are  quite  strong.  Thus  we  consider  next  a 
weaker  version.  The  Scott  family  . . .  ,a;,n_i,Co, . . . ,  cjfe_i)  :  n  <  uj}  for 

a  polynomial  time  model  A  with  universe  A  satisfying  (1)  and  (2)  is  said  to 
be  strongly  exponential  time  if  there  is  some  fixed  integer  r  >  1  such  that  the 
following  conditions  are  satisfied,  for  each  m  >  0. 

(3) '  For  any  finite  sequence  ao)  •  •  •  >am-i  of  elements  of  A,  we  can  compute  in 

time  <  r”^(maa;{2,  m,  |ao|, . . . ,  |am_i|})^  a  formula  </><  from  the  list  such  that 
®m— 1)  Co, . . .  holds  in  A 

(4)  For  each  ^^(3^0, •  •  •  ij^m  }  ^0  J  •  •  •  J  e /; — 1 )  and  each  oto ,  *  •  • ,  — 1  ^  A. ,  if 

there  exists  a  such  that  A  satisfies  . . .  ,am-i,a,co, . . .  ,cjk_i),  then 

there  exists  such  an  a  with  |a|  <  r  ■  max{r”^,  |ao|, . . . ,  |a,n-i|}- 

(5) '  For  each  0t(xo, . . .  ,a:^-i,Xm,Co, . . .  ,Cfe_i)  and  each  ao,  •  •  •  e  A,  if 

there  exists  a  such  that  A  satisfies  0t(ao, . . . , a, co, . . . , c^-i),  then  we 
can  compute  an  a  as  described  in  (4)'  in  time 
<  r^{max{2,m,  |ao|, . . . ,  la^-il}^- 

Theorem  4.  If  A  and  B  possess  a  common  strongly  exponential  time  Scott  fam¬ 
ily,  then  A  and  B  are  exponential  time  isomorphic  if  both  have  universe  Tal{uj) 
and  are  double  exponential  time  isomorphic  if  both  have  universe  Bin{u)). 

Proof.  Again  we  give  the  proof  only  in  the  case  of  a  exponential  time  Scott 
family  with  no  parameters  as  the  proof  in  the  case  with  parameters  is  essentially 
the  same.  The  isomorphism  (j)  between  A  and  B  is  given  by  the  back-and-forth 
construction  described  in  the  proof  of  Theorem  1.  It  is  easy  to  see  by  induction 
that,  for  each  m, 

a2m  <  2m,  62m+i  <  2m  -h  1,  |a2m+i|  <  and  |62m|  <  (17) 

It  follows  that,  at  stage  m  +  1,  we  can  compute  the  desired  formula  (j)t  satisfied 
by  (ao,  ■ . . ,  ttm)  or  by  (60, ... ,  bm)  in  time 

I  <  y.m+ly.(m+l)r  ^  ^^r+l^m+1 

and  we  can  then  find  the  desired  element  hm  or  in  a  similarly  bounded  time. 
Thus  the  total  time  t  required  for  stages  1, 2, . . . ,  m  -H  1  satisfies 

for  some  fixed  constant  d.  It  is  now  easy  to  see  as  in  the  proof  of  Theorem  1  that 
the  isomorphism  can  be  computed  in  exponential  time  in  tally  and  in  double 
exponential  time  in  binary.  □ 
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We  note  that  condition  (3)'  in  Theorem  4  can  be  replaced  by  the  stronger  con¬ 
dition  (3)''  that  the  list  of  all  formulas  having  m  variables  in  the  Scott  family  is 
of  length  <  r”*  and  can  be  computed  in  exponential  time  from  1”*  and  that  each 
the  statisfaction  in  A  of  each  am-iiCo, . . . ,  Ck-i)  in  the  Scott  family 

can  be  tested  in  time 


t  <  (max{2,m,  |oo|, . . . ,  (20) 

(Thus  we  can  find  the  formula  by  trying  all  formulas  in  the  list.)  For  a 
tally  universe,  condition  (5)'  then  follows  from  conditions  (3)"  and  (4)',  i.e. 
the  element  a  can  be  found  by  testing  the  satisfaction  in  A  for  all  elements  h 
where  b  <  r'^{max{2^m,  |ao|, . . . ,  |am-i|})^  of  the  formulas  of  m  -}- 1  variables 
(j)t{aQ, . . . ,  a^-i,  6)  Co, . . . ,  Ck-i)  in  the  Scott  family.  For  a  binary  universe,  there 
are  2^”*  elements  of  length  so  that  this  search  takes  too  long.  Thus  if  we 
replace  condition  (3)'  by  (3)''  and  replace  condition  (4)'  by  condition  (4)  so  that 
the  possible  length  of  am  is  exponential  in  m,  then  (5)"  would  follow  from  (3)'' 
and  (4)  since  the  search  to  be  done  in  double  exponential  time.  The  following  is  a 
result  that  can  be  proved  using  Theorem  4  in  much  the  same  way  that  Corollary 
2  was  proved  from  Theorem  1. 

Corollary  5.  Let  A  —  {B,  =^)  and  B  =  (B,=^)  be  two  polynomial  time  models 
of  an  equivalence  relation  =  such  that,  for  some  fixed  integer  k,  both  models 
satisfy  the  following: 

(i)  for  any  a  and  a*  in  the  same  equivalence  class, 


lo|'  <  A;  •  |a| 

if  B  =  Tal{uj) 

(21) 

a|'  <k+\a\ 

if  B  —  Bin{uj) 

(22) 

and 

(a)  for  any  ao, . . . ,  ftm-i  ^  B  and  any  finite  q,  if  there  is  an  equivalence  class  of 
size  q  not  containing  any  of  oq,...  ,am-i,  then  there  is  such  a  class  containing 
an  element  b  of  size 

\b\<r ‘max{r^,\ao\,.,.,\am-i\}  ifB  =  Tal{Lj)  (23) 

or 

|6|  <  (m  -f  2)*"  +  marc{ao|, . . . ,  |am-i|}  if  B  =  Bin{uj).  (24) 

Then  A  and  B  are  exponential  time  isomorphic  if  B  =  Tal{uj),  and  double 
exponential  time  isomorphic  if  B  =  Bin{uj). 

The  following  result,  which  was  proved  directly  in  [16],  follows  from  Theorem  4 
by  the  same  argument  by  which  Corollary  3  was  proved  from  Theorem  1. 

Corollary  6.  Let  A  =  {B,<^)  and  B  =  {B,<^)  be  two  dense  p-time  linear 
orderings  without  endpoints  and  possessing  linear  time  density  functions.  Then 
A  and  B  are  exponential  time  isomorphic  if  B  =  Tal{(jj)  and  double  exponential 
time  isomorphic  if  B  =  Bin{(jj). 
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Next  we  give  a  slight  modification  of  Theorem  4.  Let  us  say  that  a  . Scott  family 
is  strongly  EXPTIME  if  it  satisfies  a  slight  modification  of  the  definition  of  a 
strongly  exponential  time  family,  that  is,  we  replace  clause  (4)'  by  the  following 
clause: 

(4)"  For  each  <l)t{xo, . . .  ,Xm-i,Xm,co,  -  •  •  ,Ck~i)  and  each  ao,...,am-i  ^  if 
there  exists  a  such  that  A  satisfies  0t(ao, . . .  then 

there  exists  such  an  a  with  |a|  <  ■  max{r,  |ao|, . . . ,  |am-i|}. 

Theorem  7.  If  A  and  B  possess  a  strongly  EXPTIME  Scott  family,  then  A 
and  B  are  EXPTIME  isomorphic  if  both  have  universe  Tal{uj)  and  are  double 
exponential  time  isomorphic  if  both  have  universe  Bin{u). 

Proof.  The  modification  in  the  proof  is  that  the  size  of  Om  and  bm  is  now  of 
order  r^  ,  which  makes  the  computation  EXPTIME  rather  than  exponential 
time.  □ 

Thijs  theorem  can  be  applied  to  obtain  a  result  on  torsion  Abelian  groups.  Let 
o{a)  be  the  order  of  a  in  a  group  G.  Let  G{ao, . . . ,  ttm-i)  be  the  subgroup  of  G 
generated  by  aoj  •  •  • , ^m-i • 

Corollary  8,  Let  A  and  B  be  two  isomorphic  p-time  torsion  Abelian  groups  with 
the  same  universe  Tal{u)  such  that  for  some  fixed  integer  k, 

(i)  for  any  a,  b, 

\a  +  b\  <  k 'max{\a\,\b\}  (25) 

and 

(a)  for  any  no, ,  am-i  in  either  A  or  B  and  any  finite  q,  if  there  is  an  element 
of  order  q  not  in  G{ao, . . . ,  then  there  is  such  an  element  b  of  size 

|6|  <  (26) 

Then  A  and  B  are  EXPTIME  isomorphic  if  B  —  Tal{ijj)  and  are  double  expo¬ 
nential  time  isomorphic  if  B  =  Bin{u)). 

Proof  The  formula  (ao » •  •  •  ?  ^m)  simply  states  which  of  the  linear  combinations 
Coflo  “!“•**"{"  CjTiflTTi  —  0,"  where  e*  6  Z{q).  Since  there  are  q"^  such  formulas, 
condition  (3)'  is  clearly  satisfied.  For  condition  (4)'',  we  observe  that  if  a  G 
G{ao, . . . ,  then  we  can  compute  a  with  fewer  than  qm  additions,  so  that 

\a\  <  k^^  •  max{|do|, . . . ,  |am-i|}-  (27) 

There  are  two  cases  in  the  verification  of  condition  (5)'.  If  a  G  G(ao, . . . ,  am-i), 
then  we  can  compute  a  as  above  in  on  the  order  of  •  max{\aQ\, . . . ,  layn-il} 
steps.  If  a  is  independent  of  ao, . .  then  we  apply  clause  {ii)  and  simply 

test  all  values  for  a  with  a  <  •  max{\aQ\, . . . ,  |ajyi_i|}  □ 

We  give  one  more  general  result.  A  Scott  family  {(l)n{xi , . . . ,  Xm-i,co, . . . ,  Ck~i)  : 
n  <uj}  for  a  polynomial  time  model  A  with  universe  A  satifying  (1)  and  (2)  as 
described  above  is  said  to  be  polynomial  time  if  there  is  some  fixed  integer  r  >  1 
such  that  the  following  conditions  are  satisfied,  for  each  m  >  0. 


(3) P  For  any  finite  sequence  ao, . . .  of  elements  of  we  can  compute  in 

time  <  (maa;{2,m,  |ao|, . . . ,  |ar„-i|})^  a  formula  (j)i  from  the  list  such  that 
^t(®o  j  •  •  • }  ^m— 1>  Co  j  •  •  •  j  Cft— i)  holds  in  A^ 

(4) P  For  each  ... ,0:^-1, ... ,Cfc„i)  and  each  ao,...,ajn-i  e  A,  if 

there  exists  a  such  that  A  satisfies  (l>t(aQ,. . .  ,am-i,(i,Co,  •  ■  •  ,Ck-i),  then 
there  exists  such  an  a  with  |a|  <  (maa;{2,m,  |ao|, . . . ,  lo-m-il})^- 

(5) P  For  each  ... ,0:^-1, iCm,co, ... ,Cfc_i)  and  each  ao,...,a„i_i  G  A,  if 

there  exists  a  such  that  A  satisfies  <^t(ao, . . . , a^-i,  a, cq,  . . . , cjfe_i),  then  we 
can  compute  such  an  a  in  time  <  (7naa;{2,  m,  |ao|, . . . ,  |am-i|})^- 

Theorem  9.  If  A  and  B  possess  a  p-time  Scott  family,  then  A  and  B  are  double 
exponential  time  isomorphic  if  both  have  universe  Tal{u))  and  are  triple  expo¬ 
nential  time  isomorphic  if  both  have  universe 

The  following  corollary  of  Theorem  9  was  proved  directly  in  [16]. 

Corollary  10.  Let  A  =  (B,  <^)  and  B  =  (B,  <^)  be  two  dense  p-time  linear 
orderings  without  endpoints  and  possessing  polynomial  time  density  functions. 
Then  A  and  B  are  double  exponential  time  isomorphic  if  B  =  Tal{uj)  and  triple 
exponential  time  isomorphic  if  B  ~  Bin{uj). 
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Abstract.  Motivated  by  computer  science  challenges,  we  suggest  to  ex¬ 
tend  the  approach  and  methods  of  finite  model  theory  beyond  finite 
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1  Finite  models  and  beyond 

Although  questions  involving  finite  structures  have  always  been  of  interest  to 
logicians,  finite  model  theory  has  emerged  as  a  separate  research  area  only  in 
the  1970’s  and  early  1980’s.  Part  of  the  motivation  came  from  applications  in 
computer  science,  in  particular  from  databases  and  complexity  theory.  As  was 
pointed  out  in  [20],  finite  structures  pose  a  nontrivial  challenge  for  mathemati¬ 
cal  logic,  in  particular  for  model  theory.  Being  closely  related  to  the  foundations 
of  mathematics,  classical  logic  is  preoccupied  with  infinity.  In  fact  most  impor¬ 
tant  classical  results  and  techniques  of  mathematical  logic  (such  as  compactness, 
completeness,  the  usual  preservation  theorems)  fail  when  only  finite  structures 
are  considered.  It  was  suggested  in  [20]  that  logicians  should  systematically  de¬ 
velop  a  model  theory  of  finite  structures  that  is  able  to  cope  with  the  challenges 
from  computer  science. 

Even  though  we  believe  that  finite  model  theory  has  been  rather  successful, 
the  time  has  come  to  re-examine  the  situation.  Motivated  again  by  challenges 
from  computer  science  we  feel  that  the  strict  adherence  to  finiteness  is  too  re¬ 
strictive  and  suggest  to  extend  the  approach  and  methods  of  finite  model  theory 
beyond  finite  models. 


1.1  Motivation 

Many  of  the  finite  objects  appearing  in  computer  science  refer  at  least  implicitly 
to  infinite  structures.  In  particular,  this  is  the  case  with  objects  that  consist  of 
both  structures  and  numbers,  like  e.g.  graphs  with  weights  on  the  edges.  Such 
objects  arise  in  many  areas  of  mathematics  and  computer  science,  e.g.  in  op¬ 
timization,  databases,  complexity  theory  and  combinatorics.  Although  a  single 
such  object  may  be  representable  by  a  finite  structure,  it  is  not  always  desirable 
to  do  so.  The  numbers  appearing  in  it  live  in  an  infinite  structured  domain, 
e.g.  the  field  of  reals  or  the  arithmetic  of  natural  numbers,  and  the  arithmetical 
operations  that  we  want  to  perform  on  these  numbers  may  take  us  out  of  any 
a  priori  fixed  finite  subdomain.  Thus  it  is  desirable  to  work  directly  on  the  infi¬ 
nite  structure,  but  to  adjust  the  logical  languages  in  an  appropriate  way  so  that 
certain  complications  coming  from  the  infinity  of  the  structure  are  avoided. 

Databases.  To  explain  the  challenge  of  going  beyond  finite  models  and  inte¬ 
grating  structures  and  numbers,  we  first  look  at  database  theory,  a  particularly 
important  area  for  such  an  approach.  We  refer  to  the  books  [1,  47]  and  the  survey 
article  [34]  for  background  on  database  theory. 

The  common  practice  of  viewing  (a  state  of)  a  relational  database  as  a 
finite  structure  is  not  always  adequate;  we  are  not  the  first  to  say  that  (see 
Sect.  1.4  in  this  connection).  Let  us  look  a  little  closer  at  the  relationship  be¬ 
tween  databases  and  finite  model  theory.  In  fact,  database  theory  doesn’t  start 
with  identifying  relational  databases  and  finite  relational  structures.  Informally, 
a  relational  database  is  a  finite  collection  of  relations,  each  of  which  is  a  finite 
subset  R  C  Dx  x.  •  •  •  x  Dm  of  tuples  in  a  cartesian  product  of  domains  Du 
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the  domains  need  not  be  finite,  in  fact  it  is  often  assumed  that.  all.  domains  are 
countably  infinite.  The  active  domain  of  the  database  is  the  set  of  those  domain 
elements  that  appear  in  some  relation.  Since  the  relations  are  finite,  so  is  the 
active  domain.  So  actually,  a  database  is  a  countably  infinite  structure  all  whose 
relations  are  finite.  By  considering  the  substructure  induced  by  the  active  do¬ 
main,  a  finite  structure  is  obtained  carrying  all  the  relevant  information.  For 
many  theoretical  considerations  one  can  forget  at  this  point  where  the  domain 
elements  came  from,  and  work  with  the  finite  structure  instead. 

However,  in  real  databases  some  of  the  domains  are  not  just  plain  sets, 
but  themselves  are  (infinite)  mathematical  structures,  e.g.,  the  natural  num¬ 
bers  with  arithmetic.  Traditionally  the  relations  and  functions  structuring  these 
domains  are  not  considered  as  parts  of  the  database;  supposedly  they  are  im¬ 
posed  “from  outside” .  But  of  course,  this  additional  structure  of  the  domains  is 
used  in  database  applications.  Commercial  query  languages  like  SQL  have  arith¬ 
metical  operations  and  comparisons,  as  well  as  so-called  aggregate  functions  like 
mean,  sum,  max,  min  that  are  applicable  to  the  appropriate  domains.  In  this 
case  the  restriction  to  the  active  domain  is  no  longer  convincing,  since  arithmeti¬ 
cal  operations  may  produce  new  numbers  that  were  not  previously  stored  in  the 
database. 

We  thus  believe  that  a  more  realistic  logical  approach  to  databases  should  be 
systematically  developed,  that  does  not  adhere  to  the  strict  finiteness  condition, 
but  retains  the  essential  achievements  of  finite  model  theory. 

Discrete  dynamic  systems.  Databases  evolve  in  time  and  can  be  viewed  as 
special  discrete  dynamic  systems.  Additional  examples  are  ubiquitous  in  com¬ 
puter  science:  micro-processes,  operating  systems,  compilers,  programming  lan¬ 
guages,  communication  protocols.  Discrete  dynamic  systems  play  an  enormous 
role  in  computer  science  and  engineering.  The  problem  of  formal  specification 
of  discrete  dynamic  system  is  very  important  and  attracts  much  attention.  In 
practice,  the  most  popular  approaches  to  the  specification  problem  are  oper¬ 
ational  approaches  which  formalize  states  of  discrete  dynamic  systems  in  one 
form  or  another.  For  a  logician,  it  is  natural  to  formalize  states  as  structures  of 
first-order  logic.  This  venue  has  been  pursued  in  the  evolving  algebra  approach; 
the  venue  is  quite  practical  and  fruitful  [22]. 

Since  states  are  finite  they  can  be  formalized  as  finite  structures.  However, 
it  turns  out  that  often  it  is  more  convenient  and  practical  to  incorporate  various 
background  structures  into  states  and  deal  with  infinite  states.  This  is  a  rule, 
rather  than  an  exception,  in  the  evolving  algebra  literature  (see  [8]).  Here  we 
restrict  ourselves  to  one  simple  example. 

Imagine  that  a  state  of  interest  includes  a  stack  of  some  objects  which  may 
be  popped  or  pushed  during  the  transition  to  the  next  state.  There  are  many 
ways  to  implement  a  stack.  Respectively  there  are  many  ways  to  represent  a 
stack  in  a  finite  structure.  But  you  may  want  to  avoid  excessive  detailization, 
for  example  to  make  yoiir  verification  proof  simpler  and  cleaner.  One  solution 
is  to  have  an  auxiliary  infinite  universe  of  stacks  with  built-in  pop  and  push 
operations  and  a  nullary  function  that  gives  the  stack  of  interest  to  us.  The 
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details  of  this  simple  example  are  explained  in  the  EA  Tutorial  mentioned  in  [8] . 
More  involved  variations  of  the  example  appear  in  many  places,  in  particular 
in  Jim  Huggins’  correctness  proof  of  the  Kermit  communication  protocol  (also 
referred  to  in  [22])  where  stacks  are  replaced  by  queues. 


1.2  Metafinite  structures 

Logics  with  counting.  There  are  logics,  studied  in  the  framework  of  finite 
model  theory,  that  go  some  way  towards  integrating  logic  and  arithmetic.  These 
are  the  logics  with  counting,  augmenting  familiar  logics  like  first-order  logic  or 
fixed-point  logic  with  the  ability  to  count  the  number  of  tuples  in  any  definable 
relation.  Syntactically  this  can  be  done  by  either  counting  terms  or  counting 
quantifiers. 

The  motivation  for  considering  these  logics  comes  from  the  observation  that 
from  the  point  of  view  of  expressiveness,  first-order  logic  (FO  for  brevity)  has 
two  main  deficiencies:  It  has  no  mechanism  for  recursion  or  unbounded  iteration, 
and  it  cannot  count.  There  are  several  well-studied  logics  and  database  query 
languages  that  add  recursion  in  one  way  or  another  to  FO  (or  part  of  it),  no¬ 
tably  the  various  forms  of  fixed  point  logic,  the  query  language  Datalog  and  its 
extensions. 

On  ordered  finite  structures,  some  of  these  languages  express  precisely  the 
queries  that  are  computable  in  Ptime  or  other  complexity  classes.  However, 
this  is  not  the  case  for  classes  of  arbitrary  (not  necessarily  ordered)  structures, 
and  most  of  the  known  counterexamples  involve  counting.  Thus,  Immerman  [29] 
proposed  to  add  counting  quantifiers  to  fixed  point  logic  and  asked  whether 
this  would  suffice  to  capture  Ptime.  Although  Cai,  Purer  and  Immerman  [9] 
eventually  answered  this  question  negatively,  fixed  point  logic  with  counting 
turned  out  to  be  an  important  logic,  defining  a  natural  level  of  expressiveness 
below  Ptime,  with  a  number  of  equivalent  characterizations  [17]. 

Logics  with  counting  are  two-sorted.  With  a  one-sorted  finite  structure  21 
with  universe  A,  one  associates  the  two-sorted  structure  2t*  :=  (21,  IH)  where  = 
({0, . . . ,  n},  <))  for  n  =  \A\  and  the  canonical  ordering  <  on  {0, ... ,  n}.  The  two 
sorts  are  related  by  counting  terms  of  the  form  taking  values  in  the  second, 
numerical  sort.  The  interpretation  of  #op[p]  is  the  number  of  first-sort  elements 
a  that  satisfy  (p{a).  (Inflationary)  fixed  point  logic  with  counting  (FP  -f  C)  and 
partial  fixed  point  logic  with  counting  (PFP  +  C)  are  defined  by  closing  first- 
order  logic  under  counting  terms  and  the  usual  rules  for  building  inflationary 
or  partial  fixed  points.  The  predicates  defined  by  fixed  point  operators  may  be 
mixed,  i.e.  range  over  both  sorts.  We  refer  to  [17,  31,  40,  41]  and  to  Sect.  4.3  and 

5.3  below  for  more  background  and  results  on  fixed  point  logics  with  counting. 

It  should  be  noted,  that  although  the  second,  numerical  sort  is  of  rather  re¬ 
stricted  form  r  just  a  linear  ordering  —  this  suffices  to  define  any  polynomial¬ 
time  computable  numerical  function  in  fixed  point  logic.  Thus  it  makes  no  dif¬ 
ference  if  the  numerical  sort  has  additional  relations  and  functions,  e.g.  modular 
addition  and  multiplication,  as  long  as  these  are  polynomial-time  computable. 
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Here  we  will  consider  similar  two-sorted  structures  with  the  following  essen¬ 
tial  differences: 

-  The  numerical  sort  need  not  be  finite. 

-  The  structures  may  contain  functions  from  the  first  to  the  second  sort. 

-  We  consider  more  general  operations  than  counting. 

Metafinite  structures.  Answering  the  challenge  to  extend  the  approach  and 
methods  of  finite  model  theory  beyond  finite  models  and  integrating  structures 
and  numbers,  we  propose  here  a  more  general  class  of  structures,  which  we  call 
metafinite  structures,  and  a  number  of  logics  to  reason  about  them.  Typical 
metafinite  structures  consist  of  (i)  a.  primary  part,  which  is  a  finite  structure, 
(a)  a.  secondary  part,  which  may  be  finite  or  infinite,  and  (Hi)  a  set  of  “weight” 
functions  from  the  first  part  into  the  second.  Here  is  an  example:  a  graph,  the  set 
of  natural  numbers  with  the  usual  arithmetical  operations,  and  a  weight  function 
from  the  vertices  (or  the  edges)  of  the  graph  to  the  natural  numbers. 

By  itself,  the  notion  of  metafinite  structures  may  seem  an  old  hat.  Indeed, 
they  are  just  a  special  kind  of  two-sorted  structures.  Rather  than  just  in  the 
structures  themselves,  the  novelty  of  our  approach  is  primarily  in  the  logics  for 
such  structures,  which  access  the  primary  and  the  secondary  part  in  different 
ways. 

The  term  “metafinite  structure”  is  loose;  in  most  cases  the  secondary  part  will 
be  an  infinite  numerical  domain,  so  the  structures  are  in  fact  perfectly  infinite. 
The  term  “metafinite”  reflects  our  intention  to  apply  the  approach  and  methods 
of  finite  model  theory  to  these  structures.  In  fact  the  infinity  that  we  seek  is  very 
modest.  It  should  not  manifest  itself  too  obtrusively,  deviating  our  attention  to 
phenomena  that  are  pertinent  to  infinite  structures  only.  Therefore  our  logics  of 
metafinite  structures  —  appropriate  modifications  of  the  usual  logics  of  interest 
to  finite  model  theory,  such  as  first-order  logic,  fixed  point  logics  or  — 
access  the  infinite  part  only  in  a  limited  way,  for  instance  without  variables  (and 
therefore  without  quantifiers)  over  the  secondary  part.  An  important  feature  of 
these  languages  is  that  they  contain,  besides  formulae  and  terms  in  the  usual 
sense,  a  calculus  of  functions  from  the  primary  to  the  secondary  part,  which  we 
call  weights. 

Encoding  problems.  Of  course  one  may  object  that  also  a  weighted  structure, 
which  consists  of  both  a  structure  and  a  collection  of  numbers,  can  be  encoded 
either  by  a  pure  structure  or  by  binary  string.  This  is  true,  but  not  always 
satisfactory. 

To  encode  a  graph  with  weights  on  edges  by  a  unweighted  graph  one  could, 
for  instance,  replace  every  edge  (u,t;)  of  weight  w  hy  w  distinct  nodes,  each  of 
them  connected  to  u  and  v  but  to  no  other  nodes.  While  the  graph  obtained  in 
this  way  contains  ail  information  about  the  original  weighted  graph,  it  is  very 
inconvenient  to  perform  arithmetical  computations  on  the  encoded  weights. 

On  the  other  side,  encoding  a  structure  (with  or  without  weights)  as  a  binary 
string  requires  that  we  order  the  structure  and  thus  forces  us  to  deal  with  pre- 
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sentations  of  structures  which  contradicts  the  spirit  of  the  relational  database 
approach. 

1.3  Potential  applications 

We  have  mentioned  databases  and  discrete  dynamic  systems  as  motivations  of 
metafinite  model  theory.  There  are  numerous  other  areas  where  this  approach 
may  be  useful.  We  intended  also  to  write  a  section  on  applications  of  metafinite 
model  theory  but  his  has  to  be  deferred  to  a  later  paper.  Instead  we  mention  a 
few  things  here. 

Optimization.  Many  important  optimization  problems  are  NP-hard  and  thus 
cannot  be  efficiently  solved,  unless  P  =  NP.  One  way  to  cope  with  such  problems 
is  the  design  of  approximaiion  algorithms  which  do  not  necessarily  find  optimal 
solutions,  but  approximate  ones,  in  the  sense  that  the  quality  or  cost  of  the 
produced  solution  differs  from  an  optimal  one  only  by  constant  factor.  In  fact 
mapy  optimization  problems  admit  efficient  approximation  algorithms,  whereas 
for  others  it  has  been  shown  that  also  finding  approximate  solutions  is  NP-hard. 

Papadimitriou  and  Yannakakis  [42]  set  forth  a  new,  logical  approach  for 
studying  the  approximation  properties  of  optimization  properties.  Exploiting 
Fagin’s  logical  characterization  of  NP  by  existential  second-order  logic,  they  in¬ 
troduced  two  syntactically  defined  classes  of  maximization  problems,  Max  Snp 
and  Max  Np,  and  proved  that  all  problems  in  these  classes  admit  efficient  ap¬ 
proximation  algorithms.  The  work  of  Papadimitriou  and  Yannakakis  also  was  one 
of  the  starting  points  for  a  number  of  spectacular  non-approximability  results. 
In  particular,  the  characterization  of  NP  in  terms  of  probabilistically  checkable 
proofs,  obtained  by  Arora  et.  al.  [5],  implies  that  no  Max  SNP-hard  problem  can 
have  a  polynomial-time  approximation  scheme,  unless  P  =  NP. 

Many  optimization  problems  that  appear  in  practice  take  structures  with 
weights  as  input  instances,  e.g.  graphs  with  one  or  more  weight  functions  as¬ 
signing  numbers  to  vertices  or  edges.  Important  examples  are  the  Travelling 
Salesman  Problem,  Max  Flow/Min  Cut,  most  scheduling  problems,  and 
so  on  (see  [13]  for  additional  examples). 

As  mentioned  already  in  [42]  the  result  of  Papadimitriou  and  Yannakakis  can 
be  extended  to  problems  with  weights.  However,  the  weighted  versions  of  Max 
Snp  and  Max  Np  as  defined  in  [42]  use  the  weights  only  in  a  rather  limited  way. 
We  believe  that  metafinite  structures  provide  the  right  framework  to  extend  this 
approach  to  a  more  general  definability  theory  of  optimization  problems  with 
weights. 

Numerical  invariants  of  structures.  In  many  branches  of  mathematics,  func¬ 
tions  that  assign  numerical  parameters  to  mathematical  structures  play  an  im¬ 
portant  role.  For  instance,  a  large  part  of  graph  theory  is  devoted  to  the  study  of 
numerical  invariants  of  graphs,  such  as  genus,  chromatic  number,  clique  number, 
diameter,  girth,  etc.  Metafinite  model  theory  provides  a  framework  for  study¬ 
ing  definability  issues  of  numerical  invariants  and  relating  them,  for  instance,  to 
computational  complexity. 
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Fault-tolerance  of  queries.  Suppose  we  have  a  relational  database  where  every 
entry  has  some  probability  of  being  incorrect.  What  is  the  probability  that  the 
result  of  a  given  query  is  correct?  What  is  the  expectation  for  the  ‘‘relative 
difference”  of  the  query  applied  to  the  observed  database  with  respect  to  the 
“actual  database”. 

Such  questions  also  involve  objects  consisting  of  a  finite  structure  and  a 
collection  of  numbers.  An  unreliable  database  can  be  defined  as  a  pair  (2t, //) 
consisting  of  a  finite  structure  21  and  a  probability  function  fx  that  assigns  to 
each  each  atomic  or  negated  atomic  fact  a  probability  of  being  wrong.  With 
(21,  fi)  we  can  associate  a  probability  space  of  databases  25  with  probabilities 
z/(Q5)  to  be  understood  as  the  probability  that  the  ‘actual’  database  is  25. 

Given  a  query  Q  against  an  unreliable  database  (2t,ju),  it  is  interesting  to 
determine  its  fault-tolerance.  For  a  Boolean  query,  the  fault-tolerance  is  just  the 
probability  that  the  evaluation  against  the  observed  database  2t  gives  the  correct 
answer  for  the  actual  database  25.  For  queries  of  positive  arity,  the  fault- tolerance 
is  defined  to  be  proportional  to  the  expected  Hamming  distance  of  Q(2l)  and 
Q(25),  i.e.  the  expected  number  of  tuples  that  distinguish  between  Q(2l)  and 

Qm 

In  Sect.  3.3  we  will  show  how  to  address  these  questions  in  the  framework  of 
metafinite  model  theory.  In  particular,  we  will  prove  that  the  fault-tolerance  of 
a  conjunctive  query  is  first-order  definable. 

Note  that  we  can  also  consider  unreliable  metafinite  databases.  This  gives 
examples  where  the  secondary  part  has  itself  several  sorts,  namely  one  or  more 
sorts  for  the  numbers  appearing  in  the  database,  and  one  sort  over  the  real 
interval  [0, 1]  for  the  error  probabilities. 

Computations  over  the  real  numbers.  Blum,  Shub,  and  Smale  [7]  intro¬ 
duced  a  model  for  computations  over  the  real  numbers  (and  other  mathematical 
structures  as  well)  which  is  now  usually  called  a  BSS  machine.  It  is  essentially 
a  random  access  machine,  with  the  important  difference  that  real  numbers  are 
treated  as  basic  entities  and  that  arithmetic  operations  on  the  reals  are  per¬ 
formed  in  a  single  step,  independently  of  the  magnitude  or  complexity  of  the 
numbers  involved.  Many  basic  concepts  and  fundamental  results  of  computabil¬ 
ity  and  complexity  theory  reappear  in  the  BSS  model:  the  existence  of  universal 
machines,  the  classes  Pm  and  NPi  (real  analogues  of  P  and  NP)  and  the  exis¬ 
tence  of  NPi-complete  problems.  An  example  of  an  NP^-complete  problem  is 
the  question  whether  a  given  multivariate  polynomial  of  degree  four  has  a  real 
root. 

In  finite  model  theory  there  exist  numerous  results  relating  computational 
complexity  with  logical  definability  on  finite  structures.  The  subarea  investigat¬ 
ing  such  questions  is  sometimes  called  descriptive  complexity  theory.  The  ques¬ 
tion  arises  whether  similar  results  can  be  obtained  for  complexity  over  the  reals. 
The  main  problem  for  characterising  complexity  over  M  in  a  model-theoretic 
setting  is  to  define  the  right  class  of  structures  that  permit  a  clear  separation 
between  the  finite,  discrete  aspects  of  the  problems  and  computations  (like  in¬ 
dices  of  tuples,  time,  indices  of  registers,  the  finite  control  of  the  machines)  on 
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one  side  and  the  arithmetic  of  real  numbers  on  the  other  side. 

It  has  been  shown  by  Gradel  and  Meer  [16]  that  this  can  be  achieved  by 
M-structures,  a  special  case  of  metafinite  models,  with  the  ordered  field  of  re¬ 
als  as  secondary  part.  M-structures  admit  a  number  of  results  relating  express- 
ibility  and  complexity  that  parallel  those  of  descriptive  complexity  theory  in 
the  classical  case.  In  particular,  Gradel  and  Meer  established  analogues  to  Fa- 
gin’s  logical  characterization  of  NP  in  terms  generalized  spectra  [15],  and  to  the 
Immerman-Vardi  Theorem,  that  fixed  point  logic  captures  polynomial  time  on 
ordered  structures  [28,  48].  We  will  explain  some  of  these  results  in  Sect.  4. 

1.4  Related  approaches 

In  database  theory  there  have  been  a  number  of  proposals  for  going  beyond 
the  strict  finiteness  condition  and  incorporating  infinite  data.  In  part  this  was 
motivated  by  new  areas  of  application,  such  as  geographical  databases,  that 
involve  spatial  data.  We  mention  a  few  (by  no  means  all)  of  the  relevant  papers. 

The  study  of  infinite  recursive  structures  has  a  long  tradition  in  mathematical 
logic,  by  the  work  of  Malcev,  Nerode,  Rabin,  Vaught  (the  order  is  alphabetical) 
and  their  scientific  descendents.  Recently  there  have  been  some  papers  on  re¬ 
cursive  structures  that  study  questions  related  to  finite  model  theory.  Hirst  and 
Harel  [24]  investigated  recursive  databases,  given  by  a  finite  set  of  recursive  rela¬ 
tions  over  the  natural  numbers.  They  studied  the  notion  of  a  computable  query 
in  this  context  and  exhibited  complete  languages  for  two  specific  classes  of  recur¬ 
sive  databases.  On  the  class  of  all  recursive  databases,  quantifier-free  first-order 
logic  suffices  to  define  all  computable  queries,  whereas  a  variant  of  QL  -  the 
complete  language  from  [10]  for  the  classical  relational  model  —  is  complete  on 
highly  symmetric  recursive  databases.  In  another  paper  Hirst  and  Harel  studied 
finite  model  theory  issues,  such  as  0-1  laws  and  descriptive  complexity,  in  the 
context  of  recursive  structures  [25].  This  work  is  related  to  ours  by  the  moti¬ 
vation  to  extend  the  questions  and  methods  of  finite  model  theory  to  classes  of 
infinite  structures.  However,  metafinite  model  theory  is  radically  different  from 
recursive  model  theory. 

Kanellakis,  Kuper  and  Revesz  [35]  considered  databases  that  are  given  by 
semi-algebraic  constraints  over  the  real  (or  rational)  numbers.  This  model  can 
handle  spatial  data  and  geometric  queries  in  a  very  nice  and  convincing  way. 
Classical  relational  query  languages  can  be  extended  with  mathematical  theories 
that  admit  quantifier  elimination,  such  as  the  theory  of  real  closed  fields,  to  pro¬ 
vide  a  generalized  notion  of  query  language,  called  constraint  query  languages. 
Complexity  issues  of  such  query  languages  addressed  in  [35],  and  it  has  been 
shown- that  although  the  decision  problem  of  the  underlying  mathematical  the¬ 
ory  may  have  exponential  complexity,  the  resulting  constraint  query  languages 
admit  efficient  evaluation  algorithms.  In  this  context  we  also  refer  to  [19]  for 
some  model  theoretic  results  on  finitely  representable  databases. 

Kabanza,  Stevenne  and  Wolper  [33]  present  an  extension  of  the  relational 
database  model  for  reasoning  abount  infinite  temporal  data.  In  this  model,  time 
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is  represented  by  a  second  sort  over  the  integers  and  generalized  relations  are 
defined  by  linear  constraints,  i.e.  in  Presburger  arithmetic.  It  is  proved  that 
first-order  queries  over  such  databases  can  be  evaluated  in  polynomial  time. 

A  proposal  that  is  by  far  the  closest  to  our  approach  appears  in  the  penul¬ 
timate  section  of  the  seminal  paper  of  Chandra  and  Harel  [10],  the  same  paper 
that  also  laid  much  of  the  foundation  for  the  theory  of  computable  queries  in 
the  classical,  relational  model.  In  that  section,  Chandra  and  Harel  define  the 
notion  of  an  extended  database.  For  a  finite  domain  D  and  a  countable  infinite 
domain  F’,  an  extended  database  is  a  finite  collection  of  finite  mixed  relations  of 
the  form  R  C  x  and  functions  of  the  form  w  :  F.  Moreover  F  is 

"intended  to  include  interpreted  features  such  as  numbers,  strings  (if  needed), 
etc.” .  In  our  terminology,  an  extended  database  is  a  metafinite  structure  with 
mixed  relations.  Chandra  and  Harel  define  the  notion  of  an  extended  database 
query  and  show  that  their  language  QL  can  be  generalized  to  a  complete  query 
language  EQL  that  expresses  precisely  the  extended  computable  queries.  The 
internal  structure  of  the  secondary  part  F  is  not  really  used,  except  for  the 
assumption  that  F  is  effectively  enumerable. 

As  far  as  we  know,  this  proposal  of  Chandra  and  Harel  has  not  been  further 
pursued  in  database  theory,  in  sharp  contrast  to  the  ideas  developed  in  the  rest 
of  the  paper  [10]. 

2  Metafinite  structures 

2.1  Basic  definitions 

In  the  following,  German  letters  21,  !B, . . . ,  91, . . . ,  stand  for  finite  or  infinite  struc¬ 
tures;  their  universes  are  denoted  by  corresponding  Latin  letter  A,  H, . . . ,  R,  — 

There  are  many  variations  of  metafinite  structures.  We  define  here  three  basic 
notions: 

-  Simple  metafinite  structures. 

“  Metafinite  structures  with  multiset  operations. 

-  Metafinite  algebrcis. 

Metafinite  structures  with  multiset  operations  are  the  most  general  of  these 
notions,  and  we  will  refer  to  them  just  as  metafinite  structures.  However,  to 
simplify  the  exposition  we  start  with  the  simple  variant. 

Definition  2.1  A  simple  metafinite  structure  is  a  triple  3)  =  (21, 91,  W)  consist¬ 
ing  of 

(i)  3.  finite  structure  21,  called  the  primary  part  of  ©; 

(a)  a  finite  or  infinite  structure  91,  called  the  secondary  (or  numerical)  part 
of  We  always  assume  that  91  contains  two  distinguished  elements  0  and 
1  (or  TRUE  and  FALSE); 


^  We  denote  the  numerical  part  by  91  for  “ Ar-ithmetic” . 
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(Hi)  a  finite  set  W  of  functions  w  :  -i-  R; 

The  vocabulary  of  2)  is  the  triple  7(2))  =  (7, ,Tr,T^)  where  each  component 
of  7(2))  is  the  set  of  relation  or  function  symbols  in  the  corresponding  com¬ 
ponent  of  2).  (We  always  consider  constants  as  functions  of  arity  0.)  The  two 
distinguished  elements  0,1  of  are  named  by  constants  of  7^. 

In  finite  model  theory,  we  are  mostly  interested  in  definability  questions  con¬ 
cerning  classes  of  finite  structures.  Contrary  to  classical  model  theory,  a  single 
finite  structure  often  is  of  lesser  interest;  for  instance,  it  can  be  characterized 
up  to  isomorphism  in  first-order  logic.  Here  our  main  interest  are  definability 
questions  concerning  classes  of  metafinite  structures  with  fixed  secondary  part. 
We  write  Mr  [91]  for  the  class  of  metafinite  structures  of  vocabulary  7  with  sec¬ 
ondary  part  ^  and  Fin(7<i)  for  the  class  of  finite  structures  with  vocabulary 
Ta. 

Metafinite  structures  with  multiset  operations.  Multisets  generalize  sets 
in  the  sense  that  they  allow  multiple  occurrences  of  elements.  For  instance,  a 
function  f  :  A  defines  a  multiset  mult(/)  =  -{/(a)  :  a  G  over  R 

(the  notation  ■{. .  .J  indicates  that  we  allow  multiple  occurrences  of  elements).  A 
multiset  M  over  R  can  also  be  described  by  a  function  m  :  — >■  N  where  m(r) 
is  the  multiplicity  of  r  in  M.  For  any  set  R,  let  fm(i2)  denote  the  class  of  all 
finite  multisets  over  R, 

In  some  of  the  metafinite  structures  that  we  will  consider,  the  secondary 
part  is  not  just  a  (first-order)  structure  in  the  usual  sense,  but  comes  together 
with  a  collection  of  multiset  operations,  i.e.  operations  F  :  fm(R)  R,  mapping 
multisets  to  elements  of  R.  Natural  examples  on,  say,  the  real  numbers  are 
addition,  multiplication,  counting,  mean,  maximum,  minimum. 

Definition  2.2  A  structure  with  multiset  operations  is  a  pair  91  =  (9to,  Op) 
where  9lo  is  a  first-order  structure  and  Op  is  a  set  of  operations  F  :  fm( R) 

R  (where  R  is  the  universe  of  9to)*  The  vocabulary  7^  of  91  consists  of  the 
vocabulary  of  9lo  together  with  the  names  of  the  operations  in  Op.  A  metafinite 
structure  with  multiset  operations  is  a  triple  2)  =  (21,91,  W)  as  in  Definition  2.1 
with  the  difference,  that  91  is  a  structure  with  multiset  operations. 

Let  us  give  some  motivation  for  this  definition.  The  logics  that  we  will  con¬ 
sider  contain  formulae  and  terms.  Terms  may  take  values  in  both  parts  of  a 
metafinite  structures.  While  the  role  of  terms  over  the  primary  part  is  rather 
limited,  the  terms  taking  values  in  the  secondary  part  are  called  weight  terms 
and  are  of  crucial  importance  here. 

A  weight  term  F(xi, . . . ,  x*)  defines,  on  a  metafinite  structure  2)  =  (2t,  91,  W), 
a  function  F'^  :  A^  R.  The  collection  of  values  assumed  by  F®  forms  a  finite 
multiset 


mult(F®)  =  |F®(d)  :  a  G  A^}. 
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We  want  to  have  in  our  languages  the  expressive  means  to  apply  to  weight 
terms  natural  operations  like,  say,  summation  to  build  the  new  weight  Yin 
Algebraically,  this  means  that  we  want  to  have  operations  mapping  multisets 
over  R  to  elements  of  R.  These  multiset  operation  will  allow  us  to  build  weight 
terms  from  formulae. 

Remark.  We  consider  metafinite  structures  with  multiset  operations  as  the 
default,  and  will  usually  refer  to  them  just  as  metafinite  structures. 

Metafinite  algebras.  In  principle  we  can  always  reduce  the  primary  part  of  a 
metafinite  structure  2)  to  a  naked  set  A  by  pushing  all  the  data  into  the  functions 
in  W.  Indeed,  we  can  first  replace  every  function  /  :  >  A  by  a  (fc  +  l)-ary 

relation,  and  then  encode  every  predicate  Q  C  A^  by  its  characteristic  function 
Xq:A"->{0,1}CR. 

Definition  2.3  A  metafinite  algebra  is  a  metafinite  structure  (with  or  without 
multiset  operations)  whose  primary  part  is  a  plain  set,  i.e.  =  0.  The  elimina¬ 
tion  of  Ta-symbols  as  just  described,  associates  with  every  metafinite  structure 
2)  a  metafinite  algebra  2)“,  called  the  algebraic  form  of  2). 

As  we  will  explain  later,  the  peissage  to  metafinite  algebras  permits  a  lean 
presentation  of  a  logic  as  pure  calculus  of  terms.  In  many  cases,  this  is  convenient, 
in  others  it  is  not. 

Other  variations.  There  exist  several  other  conceivable  variations  of  metafi¬ 
nite  structures  that  are  worth  exploring.  For  instance,  instead  of  allowing  only 
functions  from  the  primary  to  the  secondary  part,  we  may  admit  mixed  relations 
P  C  A*  X  R"*  or  mixed  functions  f  :  x  R^  R.  Mixed  relations  may  be 

particularly  interesting  for  database  applications;  however,  to  allow  for  a  finite 
presentation  of  the  databases  some  restrictions  on  the  admissible  relations  have 
to  be  imposed.  A  natural  restriction  is  that  mixed  relations  be  finite  and  that 
mixed  functions  map  all  but  finitely  many  elements  to  0.  But  there  are  other 
possibilities  of  finite  presentations,  e.g.,  that  the  relations  be  recursive  [24]  or 
given  by  semi-algebraic  constraints  [35,  19]- 

We  won’t  consider  metafinite  structures  with  mixed  relations  in  this  pa¬ 
per.  However,  the  design  and  investigation  of  query  languages  for  metafinite 
databases  of  this  kind  is  one  of  the  promising  directions  for  future  research. 

Another  important  variation,  in  particular  for  databases,  are  metafinite  struc¬ 
tures  where  the  secondary  part  has  several  infinite  sortSj  e.g.  one  for  the  natural 
numbers,  one  for  strings,  one  for  real  numbers,  and  so  on.  While  this  exten¬ 
sion  poses  no  principal  difficulty,  it  often  requires  heavier  notation  and  we  won’t 
consider  such  structures  in  this  paper. 

2.2  Arithmetical  structures  and  M-structures 

Of  particular  interest  to  us  are  metafinite  structures,  whose  secondary  part  is  a 
structure  over  the  natural  numbers  (or  the  integers)  such  that  the  following 
hold: 
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-  As  a  minimum,  has  the  constants  0,1,  the  functions  +,  ♦,  the  ordering 

relation  <  and  the  multiset  operations  max,  min,  H- 

—  All  functions,  relations  and  multiset  operations  of  can  be  evaluated  in 

polynomial  time. 

Let  us  make  the  second  point  more  precise: 

Definition  2.4  Let  tTlp  be  the  structure  with  the  universe  H,  with  all  polynomial¬ 
time  computable  functions  /  :  N  (for  all  finite  arities  k)  and  with  all 

relations  i?  C  N*  (of  arbitrary  finite  arity  k)  whose  characteristic  functions  are 
polynomial-time  computable.  To  define  the  class  of  Ptime  computable  opera¬ 
tions  on  fm(N),  we  have  to  be  a  little  more  careful:  we  assume  that  multisets 
M  e  fm(N)  are  represented  by  listing  all  elements,  repeatedly  if  they  occur 
more  than  once.  Thus,  if  multM(n)  is  the  multiplicity  of  n  in  M,  the  cost  of 
M  is  ||M||  :=  XZn  Now,  OpP(N)  denotes  the  set  of  all  opera¬ 

tions  r  :  fm(H)  — »•  N  that  are  computable  in  polynomial  time  (with  respect 
to  this  representation).  PolynomiaUiime  arithmeiic,  denoted  PTA,  is  the  pair 
(^np,OpP). 

A  PTA-siruciurt  is  a  metafinite  structure  whose  secondary  part  is  PTA. 

On  the  other  hand,  we  have  as  a  ‘minimal’  variant  for  the  secondary  part  the 
structure  Oto  =  (N,  0, 1,  -|-,  •,  <,  max,  min,  X],  [])♦ 

Definition  2.5  An  arithmetical  structure  is  a  metafinite  structure  with  sec¬ 
ondary  part  such  that  9^1  is  an  expansion  of  9Io  and  a  reduct  of  PTA.  A  simple 
arithmetical  structure  is  obtained  from  an  arithmetical  structure  by  omitting  the 
multiset  operations. 

Another  interesting  class  are  M-structures,  used  by  Gradel  and  Meer  [16]  for 
developing  a  descriptive  complexity  theory  over  the  real  numbers. 

Definition  2.6  An  M-structure  is  a  simple  metafinite  structure  with  secondary 
part 

91  =  (M,  (Cr)rel)- 

It  is  convenient  to  include  subtraction  and  division  as  primitive  operations 
and  assume  that  every  element  r  G  M  is  named  by  a  constant  Cr  so  that  every 
rational  function  g  :  R*  — ►  M  can  be  written  as  a  term  (without  quantifiers). 

In  [16]  a  slightly  different  presentation  has  been  used  including  also  the  sign 
function 

f  1  if  x  >  0 

sgn(x)  :=  <  0  if  X  =  0 
I  ~1  if  X  <  0 

as  a  basic  function.  Clearly,  this  function  is  efficiently  computable,  but  is  not 
a  rational  function.  We  don’t  need  this  function  here,  because  we  have  chosen 
to  include  in  our  logics  a  characteristic  function  rule  (see  Definition  3.1)  from 
which  the  sign  function  is  easily  definable. 
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2.3  Global  functions,  numerical  invariants  and  their  complexity 

Let  /C  be  a  class  of  metafinite  structures  with  secondary  part 

Definition  2.7  A  global  function  on  /C  of  arity  Ar  is  a  function  F  that  assigns  to 
every  structure  V  =  (21,  iH,  W)  G  /C  a  (local)  function  F'^  :  R  such  that 

isomorphisms  between  structures  are  preserved:  for  every  isomorphism  h  :D  ^ 
2)'  we  have  that  for  all  ai, . . . ,  ajb  G  A 

,  at)  =  F®'(/iai, . . . ,  hat). 

In  most  cases,  SH  will  be  a  “numerical”  structure  (e.g.  the  natural  numbers 
with  arithmetical  operations,  or  the  field  of  rational,  real  or  complex  numbers) 
which  is  rigid,  thus  the  restriction  to  the  secondary  part  of  any  isomorphism 
between  structures  of  /C  is  the  identity  on  IH.  In  this  case,  we  call  a  nullary 
global  function  —  assigning  to  each  isomorphism  class  of  structures  a  numerical 
value  —  a  numerical  invariant. 

There  are  many  interesting  examples  of  numerical  invariants  both  in  the 
case  of  structures  without  weights  and  in  the  case  of  structures  with  weights: 
the  order  of  the  automorphism  group  of  the  structure,  in  graph  theory  the  usual 
graph  parameters  like  the  chromatic  number,  clique  number  or  genus,  and  also 
the  cost  of  an  optimal  solution  for  an  optimization  problem,  like  the  length  of  a 
shortest  TSP  tour.  Examples  of  global  numerical  functions  of  positive  arity  are 
the  distance  between  vertices  a:,  y  of  a  given  graph,  the  order  of  an  element  x  of 
a  given  group  (i.e.  the  cardinality  of  the  cyclic  subgroup  generated  by  x),  etc. 

Our  notion  of  global  functions  generalizes  the  notions  of  global  functions  and 
global  relations  (or  relational  queries)  in  finite  model  theory  and  databases.  Thus 
questions  concerning  computability,  complexity  and  expressibility  of  relational 
queries  on  finite  structures  can  be  viewed  as  special  cases  of  the  corresponding 
questions  on  global  functions  on  classes  of  metafinite  structures. 

Complexity  of  global  functions.  A  notion  of  complexity  for  global  functions 
may  very  much  depend  on  the  computational  model  under  consideration,  and 
on  the  cost  associated  with  the  elements  of  the  secondary  part.  For  instance,  if 
the  secondary  part  consists  of  natural  numbers  or  binary  strings,  then  we  have  a 
natural  notion  of  cost  given  by  the  number  of  bits.  On  the  other  side,  if  we  study 
complexity  over  real  numbers  with  respect  to  the  Blum-Shub-Smale  model,  then 
we  treat  every  element  of  M  as  a  basic  entity  of  cost  one. 

To  obtain  a  flexible  and  general  notion  of  complexity  of  global  functions  it  is 
therefore  convenient  to  associate  with  the  secondary  part  a  cost  function 

U:R-*n. 

The  cost  of  a  weight  function  w  :  A^  R  is  then  defined  as  ||u;||  := 
||ti;(a)||.  The  cost  of  a  metafinite  algebra  2)  =  (A,9l,  W^)  is  ||2)||  = 
iEtyeW  1 1^11  ^  metafinite  structure  can  be  defined  as  the  cost 

of  the  associated  metafinite  algebra.  Note  that  this  cost  is  always  finite,  and 
that  the  secondary  part  —  which  is  assumed  to  be  fixed  —  is  given  for  free. 
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Proviso.  For  arithmetical  structures,  we  let  ||n||  =  1  +  i-e.  the  length 

of  the  binary  representation  of  n  (with  the  convention  that  logO  =  0).  For  M- 
structures,  our  default  is  that  ||r||  =  1  for  all  r  6  M,  which  reflects  the  use  of 
M-structures  for  capturing  complexity  classes  with  respect  to  the  Blum-Shub- 
Smale  model. 

For  a  metafinite  structure  ®  =  (21,  W)  we  write  |S)|  for  the  cardinality  of 
the  primary  part  21  and  let 

max®  :=  maxmax||it;(d)|| 

w£W  a  " 

be  the  cost  of  the  maximal  weight.  Then  ||®||  <  p(|®|,max®)  for  some  poly¬ 
nomial  p(n,  m)  that  depends  only  on  the  vocabulary  of  ®.  Since  most  of  the 
popular  complexity  classes  are  invariant  under  polynomial  increase  of  the  rele¬ 
vant  input  parameters,  it  therefore  makes  sense  to  measure  the  complexity  of  a 
computation  on  a  structure  ®  in  terms  of  |®|  and  max®. 

For  instance,  an  algorithm  Af  on  a  class  C  of  metafinite  structures  runs  in 
polynomial-time  (respectively,  logarithmic  space)  if,  on  every  input  ®  e  /C,  the 
computation  of  M  terminates  in  at  most  9(|®|,max®)  steps,  for  some  polyno¬ 
mial  q,  (respectively,  uses  at  most  0(log  |®  |  +  log  max®)  of  work  space). 

More  generally,  we  can  define  the  following  notion  of  complexity 

Definition  2.8  Let  /C  be  a  class  of  metafinite  structures  with  secondary  part 
IH,  and  ||  ||  :  -->•  N  a  cost  function.  Let  Af  be  a  machine  model,  suitable  for 

evaluating  global  functions  on  C.  A  resource  measure  for  Ad  is  a  function  T 
associating  with  every  Ad-algorithm  M  and  every  input  x  a  number  € 

NU{oo}.  We  say  that  M  evaluates  the  global  F  on  C  with  resource  bound  t(n,  m) 
if,  given  any  structure  ®  E  /C,  and  any  tuple  a  of  appropriate  length  for  F*,  AT 
computes  F^(d)  in  such  a  way  that  Tm(® ,  a)  <  t(|® |,  max®). 

3  Logics  of  metafinite  structures. 

Fix  any  logic  L  (on  finite  structures),  e.g.  first-order  logic,  fixed  point  logic  or  the 
infinitary  logic  .  There  are  several  ways  to  extend  L  to  a  logic  of  metafinite 
structures. 


3.1  Simple  languages. 

The  first  such  extension,  let  us  call  it  L*  for  the  time  being,  is  suitable  for 
reasoning  on  simple  metafinite  structures.  It  is  given  by  the  following  definition. 

Definition  3.1  Let  T  =  (Ta,  T^Tw)  be  a  vocabulary  of  simple  metafinite  struc¬ 
tures  (i.e.  Tr  does  not  contain  multiset  operations).  Fix  a  countable  set  V  = 
{xo,  a?!, . . . ,  }  of  variables.  These  variables  range  only  over  the  primary  part;  we 
donH  use  variables  taking  values  in  ike  secondary  part. 

The  language  of  L*{T)  contains  the  following  expressions: 
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-  Terms  over  the  primary  part,  denoted  by  which  are. called  point 

terms.  On  a  metafinite  structure  V  =  {%  91,  W),  a  point  term  t{xiy  ...yXk) 
defines  a  function  :  A*  — >  A. 

-  Terms  over  the  secondary  part,  which  are  called  weight  terms  and  are  denoted 
by  F,  GyHy.. ..  On  iD,  a  weight  term  F{xiy  ...yXk)  defines  a  weight  function 

:A^  -^R. 

-  Formulae.  On  3),  a  formula  ,  a^jb)  defines  a  predicate  (p'^  =  {a  :V\= 

<p{a)}. 

The  terms,  weights  and  formulae  of  L*{T)  are  defined  inductively  by  the 
following  rules: 

(i )  the  set  of  point  terms  is  the  closure  of  the  set  of  V  of  variables  under 
applications  of  function  symbols  of  Ya . 

(ii)  If  <1, . . . ,  <jb  are  point  terms  and  w  is  a,  k-axy  function  symbol  of  then 
w{ti , . . . ,  tfc)  is  a  weight  term. 

(Hi)  If  Fi , . . . ,  Fjfc  are  weight  terms  and  p  is  a  ^-ary  function  symbol  of  Tr 
then  g{Fi,. .  .y  Fk)  is  a  weight  term.  In  particular,  all  all  closed  terms  (in  the 
usual  sense)  over  Yr  are  weight  terms  of  L*{Y). 

(iv)  Atomic  formulae  are  either  equalities  of  point  terms,  or  equalities  of 
weight  terms,  or  expressions  P(ti, . . .  or  Q(Fi, . . . ,  Fk)  where  P  and  Q 
are  Ar-ary  predicate  symbols  in  Ya  and  Tr,  respectively. 

(v)  The  set  of  formulae  of  L*  is  closed  under  all  rules  of  L  for  building  for¬ 
mulae.  However,  note  that  all  variables  appearing  in  these  formulae  range 
over  the  primary  part  only. 

(vi)  The  characteristic  function  rule:  If  9?  is  a  formula  of  L*,  then  xVp\  is  ^ 
weight  term  of  L* ,  with  the  same  free  variables  as  p  and  the  semantic 

xM®(a)  =  r 

^  ^  lo  otherwise. 

The  basic  terms  are  the  point  terms  and  the  weight  terms  that  can  be  built  using 
only  the  rules  (i)  -  (iv)  and  (vi).  Note  that  the  set  of  basic  terms  depends  only 
on  T,  not  on  L. 

3.2  Logics  with  multiset  operations 

We  now  turn  to  logics  that  make  use  of  multiset  operations.  As  described  in 
Sect.  2.1,  multiset  operations  can  be  used  to  define  terms  from  formulae.  They 
play  a  similar  role  as  quantifiers  do,  in  fact  quantifiers  can  be  viewed  as  a  special 
form  of  multiset  operations. 

In  the  case  where  Y^  contains  multiset  operations,  we  add  to  the  inductive 
definition  of  L*(Y)  the  following  multiset  operation  rule: 

Syntax:  Let  x  and  y  be  tuples  of  variables,  F(x,  y)  be  a  weight  terms  of  vocabu¬ 
lary  T  and  (p  he  a,  formula  of  vocabulary  T.  Then,  for  every  multiset  operation 
r  of  Tr ,  the  expression 
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G(y)  :=  rs(F(x,  ff) :  ip) 

is  a  weight  term  of  vocabulary  T,  with  free  variables  y. 

Semantic:  The  interpretation  of  G{y)  on  an  T-structure  S)  with  valuation  b  for 
y  is 

G®(6):=r|F®(a,6):®|=^(a,6)J. 

To  enhance  readability,  we  will  sometimes  omit  the  free  variables  and  use 
the  abbreviated  notation  :  (p).  Furthermore,  we  will  omit  p  in  case  it  is  a 
tautology  and  write  r$F{x^  y). 

There  are  some  important  multiset  operations  that  are  invariant  under  adding 
arbitrary  occurrences  of  0  to  the  multiset:  r(5)  =  r(5U  t0,0, . .  .,0J)  for  all 
5  E  fm(iJ).  For  instance,  this  is  the  case  for  and  max.  In  this  case,  we  can 
use  {Fs  F  •  xM)  rather  than  Fs^F  :  p). 

Example  3.2  [Binary  representations]  Consider  arithmetical  structures  with 
primary  part  of  the  form  21  =  ({0, . . . ,  n  -  1},  <,  P)  where  P  is  a  unary  relation. 
P  is  interpreted  as  a  bit  sequence  uo  ’  ’  •'Wn-i  representing  the  natural  number 
Wt2*  (where  «,■  =  1  iff  21  [=  P{i)).  The  number  represented  by  P  is  definable 
by  the  term 

n(2  =  y  <  ®))- 

X  y 

Example  3.3  [Counting  elements]  On  arithmetical  structures,  we  can  count  in 
FO*.  For  any  formula  there  is  a  weight  term  #x[y>(^)]  counting  the  number 
of  tuples  a  such  that  (p{a)  is  true.  Indeed, let 

#*b(®)]  := 

Example  3.4  [Counting  equivalence  classes]  Let  ®  be  an  arithmetical  structure 
and  (p{x,  y)  be  a  binary  formula,  defining  an  equivalence  relation  on  A,  If  we 
have  division  as  a  basic  function  in  %  then  the  index  of  denoted  #[A/(p], 
is  definable  in  FO*  in  the  following  way. 

By  the  previous  example,  F{x)  =  #y[(p{x,y)]  is  a  weight  term  of  FO*.  The 
index  of  can  be  written  as  a  sum  of  rational  numbers:  if^[A/(p]  =  ^^{F{x))~^ . 
To  do  everything  over  N,  let  G  =  n®  f^us  the  weight  G/P(a;)  is  also  FO*- 
definable  and  we  get 

*W<p]  =  {J2G/Fix))/G. 

X 

Multiset  operations  play  an  important  role  in  metafinite  model  theory.  They 
partially  compensate  for  the  limited  access  to  the  secondary  part  and  greatly 
enhance  the  expressive  power  of  the  logics  that  we  consider.  We  also  believe 
that  they  provide  the  right  logical  formalism  for  the  aggregate  operators  used  in 
databases  (see  Chapter  7.3:  “Confronting  the  Real  World”  in  [1]). 
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3.3  An  excursion:  Reliability  of  queries 

We  present  here  a  more  elaborate  example  for  the  use  of  multiset  operations 
that  addresses  the  issue  of  fault  tolerance  of  relational  queries  as  mentioned  in 
Sect.  1.3. 

Definition  3.5  An  unreliable  database  is  a  pair  where  21  is  a  finite  struc¬ 

ture  and  a  probability  function  on  the  of  atomic  statements  R{a)  about  2t. 

Think  about  2t  as  the  observed  database.  For  every  first-order  statement  ^(d) 
about  2tj  let  Wrong(^(d))  be  the  event  that  the  truth-value  of  (p{a)  in  21  differs 
from  the  truth-value  of  (p{a)  in  the  actual  database.  fi{R(a))  is  the  probabil¬ 
ity  of  the  event  Wrong(JJ(d)).  It  is  supposed  that  the  events  Wrong(jR(d))  are 
independent. 

Let  ©  be  a  database  of  the  same  vocabulary  as  21  and  with  the  same  universe 
as  2t.  Let  D(!8)  be  the  collection  of  atomic  statements  i2(d)  that  are  true  in  23. 
The  probability  that  23  is  the  actual  database  is 

v{<B)  ~  JJ  J][  (1  - 

V>€|D(a)-D(<B)|  v>€B(SI)09€D(S) 

where  |D(2t)  —  D(23)|  is  the  symmetric  difference. 

Given  a  relational  query  of  arity  k,  let  =  {d  G  A*  :  2t  ]=  'tp{a)}. 
The  Hamming  distance  between  and  is  the  cardinality  of  the  symmetric 
difference  —  '0®|- 

Definition  3.6  With  every  tuple  d  6  we  associate  the  random  variable 
Pxp{a)  that  assigns  to  a  database  ®  the  probability  that  ^(d)  distinguishes  be¬ 
tween  23  and  the  observed  database  21.  Summing  up  the  probabilities  Pxp(a)  over 
all  tuples  d  G  A^  gives  the  expectation  of  the  Hamming  distance  between 

and  V’®*  The  number  :=  1  -  [E{Hrp)/n^]  is  the  faulUiolerance  of  xl). 

For  simplicity,  we  restrict  attention  to  conjunctive  queries,  that  is  to  queries 
of  the  form 


3xi..3xk[(piA--A(pi] 

where  each  (pi  is  an  atomic  or  negated  atomic  formula.  It  is  supposed  that  the 
atomic  parts  of  formulas  (pi  and  pj  are  different  if  i  ^  j. 

Unreliable  databases  can  be  modelled  by  metafinite  structures  where  the 
secondary  part  9^  is  the  field  of  reals  with  the  multiset  operations  X),  fl*  check 
that  the  expected  Hamming  distance  and  the  fault- tolerance  of  the  conjunctive 
queries  are  first-order  definable  numerical  invariants. 

Let  (21, /i)  be  an  unreliable  database  of  relational  vocabulary  T^.  View  /i  as  a 
tuple  of  probability  functions  fiR  where  i2  is  a  proper  predicate  (not  the  equality 
sign)  in  Ta  (and  fjiR  is  the  restriction  of  /z  to  atoms  of  the  form  i2(d).  With  an 
unreliable  database(21,  fi),  we  associate  the  metafinite  structure  (21,  IH,  {jjIr  :  R  € 
Ta}). 
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Proposition  3.7  Let  ip{x)  be  a  subformula  of  a  conjunctive  query  as  above. 
Then 

(i)  Pfp{x)  is  a  first-order  definable  global  weight  function; 

(a)  The  expected  Hamming  distance  and  the  fault-tolerance  of  ij)  are  first- 
order  definable  numerical  invariants. 

Proof.  Since  E{Hg,)  =  =  1  —  the  second  claim 

follows  from  the  first. 

The  first  claim  is  proved  by  induction  on  xp(x).  If  ^  is  an  atom  R{x)  then 
clearly  Pip{x)  =  fiR{x)  which  is  also  atomic.  Obviously  P^g,{x)  =  Pip{x). 

Let  N(xi^ . . . ,  ajjb)  be  the  assertion  that  Xi  ^  xj  if  i  <  j,  and  let  iV(^(x))  = 
N(x).  It  suffices  to  find  a  weight  term  that  expresses  P^p  only  in  the  case  when 
A^(^)  hold.  Indeed,  let  ai, . . . ,  am  be  all  different  consistent  assertions  about  the 
equality  relation  on  the  components  of  x.  P^  is  the  sum  of  probabilities  PoaAxpy 
and  each  ai  twj)  is  equivalent  to  a  formula  of  the  form  N{y)  A  ^{y). 

Now  suppose  that  is  a  conjunction  V^i(x)  A'02(^)  where  x  =  (iCi, . . . ,  ajjt) 
and  restrict  attention  to  the  case  when  N(x)  holds.  The  events  Wrong(^i(d)) 
are  independent. 

In  case  21  |=  ^(d),  Wrong(V'(a))  =  Wrong(V’i (a))  U  Wrong(^2(d)).  In  case 
21  ^  (-iV’t(d)  A  V^3_i(d)),  Wrong(^(d))  =  Wrong(^,(d)).  Finally,  in  case  21 
(-«^i(d)  A  -1^2(0),  Wrong(V’(a)  =  Wrong(^i(d)  fl  Wrong(^2(a))- 

Recall  that  the  characteristic  function  xiV’]  of  first-order  formula  is  a 
first-order  weight  term.  We  have: 


Prp  =  x[m-{^-Pn>.){i-P^.)] 

-f  xhV’i  A  i)2]Pg>^ 

+  x[^i  A 

+  xh^  A -i^2]F,/-iF^2 

Finally,  suppose  that.'0(^)  =  (32/)(^(x,  y))  and  restrict  attention  to  the  case 
that  N{x)  holds.  It  suffices  to  find  a  weight  term  that  expresses  the  probability 
V  =  P\l>{x)  in  the  case  that  that  N{x^  y)  holds.  Indeed,  fix  an  instantiation  d  of  ^ 
such  that  the  components  ai, . . . ,  ajb  of  d  are  distinct.  F^(a)  is  the  sum  pf  p  and 
the  probabilities  Pip{a,ai)'  Now  use  the  induction  hypothesis. 

Restrict  attention  to  the  case  that  N{x,y)  holds.  Without  loss  of  generality, 
every  conjunct  of  (p  contains  y.  Indeed,  let  a{x,  y)  be  the  quantifier-free  part 
of  y?,  and  P{x)  be  the  conjunction  of  those  conjuncts  of  a  that  do  not  contain 
y.  Clearly,  is  equivalent  to  P{x)  A  (3y)a(x,y).  Furthermore,  the  events 
Wrong(/?(d)  and  Wrong((3y)a(x,  y))  are  independent  and  thus  the  conjunction 
can  be  treated  as  above.  Now  use  the  induction  hypothesis. 

Again,  fix  an  instantiation  d  of  ^  such  that  the  components  ui, . . . ,  ajfe  of  d 
are  distinct.  Since  every  conjunct  of  p  contains  y,  statements  9p(d,6),  where  b 
ranges  over  A  —  {oi, . . . ,  a^},  are  pairwise  independent. 
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In  case  2t  j=  tp{a),  Wrong(V’)  =  Wrong(^(a,  6)  where  B  is  the  collection 
b  €  A~  {ai,...,ak}  such  that  Ql  \=  <p{a,  6).  In  case  Ql  |=  Wrong(V^(a))  = 

(Jj  Wrong(9?(a,  6)).  We  have 


p>t> = xM  •  S')  ^  S'))  + 


□ 


3.4  Pure  term  calculi 

We  now  explain  how,  for  metafinite  algebras,  logics  can  be  presented  as  pure 
calculi  of  weight  terms.  We  first  assume,  for  simplicity,  that  the  secondary  part 
91  is  an  algebra,  i.e.  the  vocabulary  Tr  contains  no  relation  symbols.  Thus  we 
deal  with  vocabularies  T  =  (Tr,Tu,)  consisting  of  two  sets  of  function  symbols. 

Definition  3.8  The  calculus  FOT(r)  of  first-order  terms  of  vocabulary  T,  to¬ 
gether  with  the  notion  of  the  rank  of  a  term,  is  defined  inductively  as  follows: 

(i)  Ifa:i,...,a:jb  are  variables,  and  u;  is  a  k-aiy  weight  function  in  T,/;,  then 
w(xi , . . . ,  Xm)  is  a  term  of  rank  0  in  FOT(T). 

(ii)  If  Fi, Fm  ^  FOT(T)  and  g  e  Tr  is  a>  m-ary  function  symbol,  then 
g{Fi , . . . ,  Fm)  is  a  term  of  FOT(T),  whose  rank  is  the  maximum  of  the  ranks 
of  F’l ,  .  .  . ,  Fm- 

(Hi)  If  F  and  G  belong  to  FOT(T),  then  so  does  x[jF’  =  G\.  The  rank  of 
x[F  =  G]  is  the  maximum  of  the  ranks' of  F  and  G. 

(iv)  If  F  and  G  belong  to  FOT(T),  y  is  an  ^-tuple  of  variables  and  F  a 
multiset  operation  from  then  ry{F(x,  y)  :  G  =  1)  is  a  term  of  FOT(T), 
of  rank  i  -{-  max{rk(F),  rk(G)}. 

For  terms  formed  with  multiset  operations,  we  also  use  a  simplified  form 
Fy  F(x,y)  as  an  abbreviation  for  {Fy  {F{x,y)  :  1  =  1). 

It  is  clear  that  for,  say,  arithmetical  structures,  or  for  M-structures  with  max¬ 
imization  as  a  multiset  operation,  first-order  logic  can  be  simulated  by  FOT,  in 
the  sense  that  the  characteristic  function  of  every  first-order  formula  is  equiva¬ 
lent  to  a  term  in  FOT.  Indeed,  this  follows  by  a  straightforward  induction  using 
the  following  equalities: 

x[i>  A  9?]  =  x['4’]x[<f>] 
xhV’]  =  1  -  xM 
x[3xV’]  =  max(x[V’]). 

X 

In  fact,  this  holds  for  all  secondary  parts  as  long  as  we  have  two  definable 
functions  A  and- interpreted  on  {0, 1}  C  ii  in  the  usual  way,  and  any  multiset 


332 


operation  that  distinguishes,  say,  multisets  with  occurrences  of  1  from  those 
without  (or  the  empty  multiset  from  the  nonempty  ones). 

Remark.  The  restriction  to  algebraic  structures  is  not  necessary.  When  we  deal 
with  an  arbitrary  vocabulary  T  =  (Ta^Tr^T^t,)  for  metafinite  structures,  we  can 
still  present  first-order  logic  as  a  pure  calculus  of  weight  terms.  We  just  have  to  re¬ 
place  in  clause  (i)  of  Definition  3.8  the  variables  Xi  by  arbitrary  point  terms  over 
Ta  (as  in  clause  (ii)  of  Definition  3.1),  and  add  the  rules  defining  for  every  pred¬ 
icate  Q  and  already  defined  terms  Fi, , . . ,  also  the  term  x[Q(Fi, . . . ,  F^)]. 

3.5  Second-order  multiset  operations 

In  several  contexts,  for  instance  for  dealing  with  NP-optimization  problems  or 
with  counting  problems  in  the  class  #P,  it  is  convenient  to  have  logics  with 
second-order  constructs. 

Multiset  operations  can  be  viewed  as  a  generalization  of  quantifiers.  There¬ 
fore,  natural  variants  of  second  order  logics  can  be  defined  by  applying  multiset 
operations  to  predicate  variables. 

Definition  3.9  Suppose  we  have  a  logic  L  in  the  usual  sense  (say,  second-order 
logic  or  it  existential  fragment  Tf),  then  L**  is  the  smallest  logic  closed  under 
the  rules  of  L*  together  with  the  following  rule. 

Multiset  operation  rule  (second  order): 

Syntax:  Let  T  -  (Ta.Tr.Tyj)  he  a,  vocabulary  and  T  =  (To  U  {X},  r«;)  where 

X  is  a  tuple  of  relation  variables.  If  F  is  a  weight  term  and  (p  a  formula  of 
vocabulary  T'  with  free  variables  among  x,  y,  then,  for  every  multiset  operation 
r  of  Tr,  the  expression 

is  a  weight  term  of  vocabulary  T,  with  free  variables  y. 

Semantic:  The  interpretation  of  this  expression  on  an  T-structure  S)  with  valu¬ 
ation  6  for  y  is 

6)  :  (X,  a)  satisfy  (2),  X)  |=  ^p(o,  6) J. 

Example  3.10  [The  Travelling  Salesman  Problem]  NP-optimization  problems 
like  the  TSP  can  be  expressed  in  a  very  direct  way  in  this  framework,  since  the 
arithmetic  that  is  necessary  to  determine  the  length  of  a  tour  and  to  minimize 
is  separated  from  the  graph. 

Let  drder(<)  express  that  <  is  a  linear  ordering,  and  let  succ(<,  ic,  ?/)  be  a 
formula  which,  for  any  given  linear  ordering  <,  says  that  either  y  is  the  successor 
of  a;,  or  a:  is  the  maximal  and  y  the  minimal  element  of  the  ordering.  Then  the 
length  of  the  shortest  tour  of  any  instance  (V,  w)  of  the  TSP,  where  w  :  VxV  —>■ 
N  is  the  weight  function  giving  the  distances,  is  defined  by  the  weight 
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optTSp{Vi  ti;)  =  y)  :  succ)  :  order^ 


x,y 


A  more  challenging  example:  the  genus  of  a  graph.  The  genus  7(G)  of 
an  undirected  graph  G  is  the  smallest  5  €  N  such  that  G  can  be  embedded  into 
the  sphere  with  g  handles. 

The  genus  is  one  of  the  most  important  graph  parameters.  It  is  hard  to 
compute;  the  corresponding  decision  problem  —  given  a  graph  G  and  a  number 
k,  decide  whether  7(G)  <  k  —  is  NP-complete. 

It  is  more  convenient  for  us  to  work  with  a  different,  purely  combinatorial 
characterization  of  the  genus. 

Definition  3.11  A  rotation  system  on  a  undirected  graph  G  =  (V,  E)  is  a 
ternary  predicate  P  £  which  defines  for  every  node  a  cycle  on  the  edges 
incident  to  it.  More  precisely:  if  {x,y,z)  G  P,  then  {x,y)  G  E  and  {y,z)  G  P, 
and  for  sAly  eV,  the  directed  graph  Hy  =  {Sy,Cy)  with 

Sy:-{x:{x,y)€E} 

Cy  :=  {{x,z)  :  (x,y,z)  G  P} 

is  a  cycle.  A  Efface  is  defined  by  a  cycle  xq^  . . .  ,Xr-i  in  G  such  that,  for  all 
i  <  r,  ajj,  x,+i)  G  P  (here,  indices  are  expressed  modulo  r).  The  P~genus 

of  G,  denoted  7(P)  is  defined  by  Euler’s  formula 

n-e  +  /(P)=:2-27(P) 

where  n  is  the  number  of  vertices,  e  the  number  of  edges  and  /(P)  the  number 
of  P-faces. 

The  following  result  is  well-known  in  graph  theory  (see  e.g.  [18]) 
Proposition  3.12  The  genus  of  G  is  the  minimal  P-genus  of  G. 

For  convenience  our  logical  definition  of  the  genus  is  based  on  transitive 
closure  logic.  This  is  a  familiar  logic  in  finite  model  theory  which  augments  first- 
order  logic  by  the  ability  to  define  transitive  closures.  It  admits,  for  every  formula 
(p{x,y)  with  fc-tuples  of  free  variables,  also  the  formula  [TC^,y  ex¬ 

pressing  that  (d,6)  is  contained  in  the  reflexive  and  transitive  closure  of  the 
binary  relation  that  defines  on  fc- tuples. 

It  is  easy  to  see  that  there  exists  a  formula  '(p  of  vocabulary  {E,  P}  in  tran¬ 
sitive  closure  logic  such  that  for  every  graph  G  and  every  ternary  predicate  P 
on  G 

(G,  P)  1=  V’  if  and  only  if  P  is  a  rotation  system  on  G. 

The  number  of  P-faces  is  the  number  of  equivalence  classes  of  directed  edges 
with  respect  to  the  reachability  relation  defined  by  P.  It  is  not  difficult  to  con¬ 
struct  a  formula  a(P,  Q)  in  transitive  closure  logic  saying  that  Q  is  a  binary 
relation  containing  at  most  one  directed  edge  on  each  P-face: 
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a;(P,  Q)  =  'ix'iy'iu^v{{Qxy  A  Quv  A. 

[TCj,j,,u„j/  =  «  A  P(x,  y,  t))](xy, ««))  -+  (x,  y)  =  {u,  v)). 

Given  that  'tp(P)  expresses  that  P  is  a  rotation  system,  that  the  weight 
is  definable  in  FO*({Q})  and  that  n  and  e  are  obviously  definable,  we  can  define 
the  genus  of  an  undirected  graph  by 

7  =  1  +  -  max(#Q  :  V>  A  a)). 

4  Descriptive  complexity 

One  of  the  goals  of  metafinite  model  theory  is  the  descriptive  complexity  theory 
of  problems  with  weights.  For  finite  models,  the  results  of  Fagin,  Immerman, 
Vardi  and  others  provide  logical  characterizations  of  NP,  P  and  also  for  most  of 
the  other  important  complexity  classes,  at  lezist  on  ordered  structures.  We  refer 
to  [21,  29,  30]  for  surveys  on  descriptive  complexity. 

Here  we  investigate  generalizations  of  these  results  in  the  realm  of  metafinite 
structures.  For  simplicity,  we  focus  on  arithmetical  structures;  we  also  mention 
M-structures  but  refer  to  [16]  for  proofs.  However,  the  approach  can  be  extended 
to  problems  on  metafinite  structures  with  arbitrary  secondary  part.  This  requires 
the  definition  of  a  suitable  machine  model  and  a  suitable  notion  of  complexity. 
We  will  defer  the  detailed  development  to  a  subsequent  paper. 

We  start  with  the  observation  that  first-order  logic  can  be  evaluated  in 
polynomial-time. 

Proposition  4.1  If  the  basic  functions,  relations  and  multiset  operations  of 
can  be  evaluated  in  polynomial  time  (with  respect  to  the  given  cost  function), 
then  the  same  is  true  for  every  first-order  definable  global  function  on  Mr[9l]. 

The  proof  is  a  straightforward  induction. 

4.1  Metafinite  spectra 

We  first  consider  Fagin ’s  characterization  of  NP  by  existential  second-order  logic 

[15]. 

Definition  4.2  [Fagin]  A  class  JC  of  finite  Ta-structures  is  a  generalized  spectrum 
if  there  exists  a  first-order  sentence  ‘tp  of  a.  vocabulary  Ta  U  {i^i, . . . ,  Rm}  such 
that  21  G  /C  if  and  only  if  there  exists  an  expansion  05  of  21  with  \=  ip. 

Remark.  An  equivalent  definition  is  that  a  generalized  spectrum  is  the  class 
of  finite  models  of  an  existential  second-order  sentence  3Ri  •  •  •  3Rm'^-  However, 
as  discussed  below,  there  are  several  possibilities  of  generalizing  second-order 
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logic  to  metafinite  structures,  and  we  don’t  want  to  commit  ourselves  to  one 
particular  variant.  We  will  therefore  mostly  work  with  (generalizations  of)  the 
definition  given  above. 

Informally,  Fagin’s  Theorem  states  that  the  generalized  spectra  are  precisely 
the  model  classes  recognizable  in  nondeterministic  polynomial  time.  For  a  precise 
statement  of  this  result,  we  have  to  keep  in  mind  that  to  serve  as  an  input 
for  a  classical  computational  device  like  a  Turing  machine,  a  finite  structure 
needs  to  be  encoded  by  a  string.  At  least  implicitly,  such  an  encoding  requires 
that  an  ordered  representation  of  the  structure  is  chosen.  The  precise  form  of 
the  encoding  is  not  important,  as  long  as  it  satisfies  some  reasonable  simple 
properties.  So  when  we  say  that  a  class  of  structures  is  in  NP  we  actually  mean 
that  the  set  of  encodings  of  structures  in  that  class  is  in  NP. 

Theorem  4.3  (Fagin)  Lei  K  be  a  class  of  finite  structures  of  a  fixed  finite 
vocabulary  which  is  closed  under  isomorphisms.  Then  JC  is  in  NP  if  and  only  if 
it  is  a  generalized  spectrum. 

Does  Fagin’s  Theorem  generalize  to  metafinite  structures?  To  address  this 
problem,  we  need  to  make  precise  two  notions: 

-  The  notion  of  a  metafinite  spectrum,  i.e.  a  generalized  spectrum  of  metafinite 
structures. 

“  The  notion  of  nondeterministic  polynomial  time  complexity  in  the  context 
of  metafinite  structures. 

We  start  with  two  notions  of  metafinite  spectra. 

Definition  4.4  A  class  JC  C  Mr  [91]  is  a  metafinite  spectrum  if  there  exists  a 
first-order  sentence  ^  of  a  vocabulary  T'  D  T  such  that  JD  G  /C  if  and  only  if 
there  exists  an  expansion  S)'  G  Mr'[fH]  of  S)  with  ^  A  primary  metafinite 
spectrum  is  defined  in  a  similar  way,  except  that  only  the  primary  part  of  the 
structures  is  expanded.  This  means  that  the  expanded  structures  D*  have  the 
same  set  of  weight  functions  as 


Remark.  These  two  notions  of  metafinite  spectra  correspond  to  two  variants 
of  (existential)  second-order  logic.  The  more  restrictive  one  allows  second-order 
quantifiers  only  over  primary  relations,  whereas  the  general  one  allows  quan¬ 
tification  over  weight  functions  as  well.  Thus,  a  primary  metafinite  spectrum  is 
the  class  of  models  T)  G  My  [9^]  which  are  models  of  an  existential  second-order 
sentence  of  the  form  3Ri  •  ♦  •  3Rm^  where  i2i, . . . ,  Rm  are  relation  variables  over 
the  primary  part  and  tp  is  first-order  (in  the  sense  of  Definition  3.1).  Since  rela¬ 
tions  over  the  primary  part  can  be  replaced  by  their  characteristic  functions,  a 
metafinite  spectrum  in  the  more  general  sense  is  the  class  of  models  of  a  sentence 
3Fi  •  •  '3Fm'fp  where  Fi  are  function  symbols  ranging  over  weight  functions. 
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4.2  Generalizations  of  Fagin’s  Theorem 

We  show,  that  both  notions  of  metafinite  spectra  capture  (suitable  variants  of) 
nondeterministic  polynomial-time  in  certain  contexts,  but  fail  to  do  so  in  others. 

First  we  consider  arithmetical  structures  where  the  secondary  part  is 
as  given  by  Definition  2.5.  We  assume  that  that  the  cost  of  natural  numbers  is 
given  by  the  length  of  their  binary  representations.  As  described  in  Sect.  2,3,  this 
gives  a  natural  notion  of  the  complexity  of  global  functions,  and  in  particular 
of  an  NP-class  of  arithmetical  structures.  So  the  question  is,  whether,  or  under 
what  circumstances  NP  is  captured  by  the  class  of  metafinite  spectra  or  primary 
metafinite  spectra. 

The  original  proof  of  Fagin’s  Theorem  generalizes  to  the  case  of  arithmetical 
structures  with  not  too  large  weights. 

Definition  4.5  A  class  K  of  metafinite  structures  has  small  weights  if  there 
exists  a  ^  €  N  such  that  max©  <  |©|^  for  all  ©  G  /C. 

Recall  that  max©  stands  for  the  cost  of  the  largest  weight.  Thus,  a  cletss 
of  arithmetical  structures  has  small  weights  if  the  values  of  the  weights  are 
bounded  by  a  function  2^(1'^!^  for  some  polynomial  p.  We  obtain  the  following 
first  generalization  of  Fagin’s  result. 

Theorem  4.6  Let  K  C  Mr  [01]  be  a  class  of  arithmetical  structures  with  small 
weights,  which  is  closed  under  isomorphisms.  The  following  are  equivalent: 

(i)  K  is  in  NP. 

(ii)  JC  is  a  primary  generalized  spectrum. 

Proof.  It  is  obvious  that  (ii)  implies  (i).  The  converse  can  be  reduced  to  Fagin’s 
Theorem  as  follows.  We  assume  that  for  every  structure  ©  =  (21,01,  W)  in  JC, 
we  have  that  max©  <  n*  where  n  =  |©|  =  |2t|;  further  we  suppose  without 
loss  of  generality  that  an  ordering  <  on  A  is  available  (otherwise  we  expand  the 
vocabulary  with  a  binary  relation  <  and  add  a  conjunct  ^(<)  asserting  that  < 
is  a  linear  order).  We  can  then  identify  with  the  initial  subset  {0, . . . ,  n*  -  1} 
of  N,  viewed  as  bit  positions  of  the  binary  representations  of  the  weights  of  ©. 
With  every  ©  G  /C  we  associate  a  finite  structure  ©y  by  expanding  the  primary 
part  Qt  as  follows:  For  every  weight  function  it;  G  W  of  arity  j  we  add  a  new 
relation  P^)  of  arity  j  +  k  with 

:=  {(d,t)  :  the  i-th.  bit  of  it;(d)  is  1}. 

Then  JC  is  in  NP  if  and  only  if  /C/  =  {©/  :  ©  G  /C}  is  an  NP-set  of  finite 
structures,  and  in  fact,  we  can  choose  the  encodings  in  such  a  way  that  ©  and  © y 
are  represented  by  the  same  binary  string.  Thus,  if  JC  is  in  NP,  then  by  Fagin’s 
Theorem  /Cy  is  a  generalized  spectrum,  defined  by  a  first-order  sentence  i/j. 

As  in  Example  3.2,  one  can  construct  a  first-order  sentence  a  (whose  vocab¬ 
ulary  consists  of  the  weight  functions  it;  G  and  the  corresponding  primary 
relations  P^),  which  expresses  that  the  P^,  encode  the  weight  functions  w  in  the 
sense  defined  above.  Then  ^  A  a  is  a  first-order  sentence  witnessing  that  /C  is  a 
primary  metafinite  spectrum.  □ 
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Remark.  The  same  result  holds  for  simple  arithmetical  structures. 

However,  without  the  restriction  that  the  weights  be  small,  it  is  no  longer 
true  that  every  NP-set  is  a  primary  metafinite  spectrum.  If  we  have  inputs  with 
huge  weights  compared  to  the  primary  part,  then  relations  over  the  primary  part 
cannot  encode  enough  information  to  describe  computations  that  are  bounded 
by  a  polynomial  in  the  length  of  the  weights. 

It  is  tempting  to  use  unrestricted  metafinite  spectra  instead.  However,  metafi¬ 
nite  spectra  in  the  general  sense  capture  a  much  larger  class  than  NP. 

First,  we  note  that  any  tuple  a  ^  N*  can  be  viewed  as  an  arithmetical 
structure  with  the  empty  primary  primary  vocabulary  and  k  weight  functions 
ai, . . . ,  ajfc  which  happen  to  be  nullary.  Thus  an  arithmetical  relation  5  C  N*  can 
be  viewed  as  a  special  class  of  arithmetical  structures. 

Theorem  4,7  Every  recursively  enumerable  set  5  C  is  a  metafinite  spec¬ 
trum.  In  particular,  there  exist  undecidable  metafinite  spectra. 

Proof.  By  Matijasevich’s  Theorem  (see  [38])  every  recursively  enumerable  set 
5  C  N*  is  Diophantine,  i.e.  can  be  represented  as 

5  =  {d  G  :  there  exists  6i , . . . ,  6^  G  N  such  that  Q(d,  6)  =  0} 

for  some  polynomial  Q  G  Z[xi, . . .  ,Xk,yi, . . .  ,ym]-  Let  P,P'  G  H[x,  y]  such 
that  Q{x,y)  =  P{x,  y)  —  P'{x,  y)  Thus  5  is  a  metafinite  spectrum;  the  desired 
first-order  sentence  uses  additional  weight  functions  6i , . . . ,  6^  and  asserts  that 
P(a,b)  =  P\a,b).  □ 

This  can  be  extended  to  any  r.e.  class  of  arithmetical  structures,  with  ar¬ 
bitrary  vocabulary.  To  prove  this,  we  describe  how  to  encode  structures  V  C 
Mrpl]  by  tuples  c(2))  G  where  k  depends  only  on  T.  (In  fact,  it  is  no  prob¬ 
lem  to  reduce  k  to  1.)  For  future  use  of  such  encodings  we  will  be  more  restrictive 
than  necessary  for  this  result. 

Similar  to  the  case  of  finite  structures,  an  encoding  involves  the  selection  of 
a  linear  order  on  the  primary  part.  In  fact  we  find  it  more  convenient  to  have  a 
ranking  of  the  primary  part  rather  than  just  a  linear  ordering. 

Definition  4.8  Suppose  that  SH  contains  a  copy  of  (N,  <).  A  ranking  of  a  metafi¬ 
nite  structure  5)  =  (21, W)  is  a  bijection  r:A— ►  {0,...,n— 1}  C  R.  A  class 
JC  C  Mr  [9^1]  is  ranked  if  T  contains  a  weight  function  r  whose  interpretation  on 
every  S)  G  /C  is  a  ranking. 

From  a  ranking  one  can  trivially  define  a  linear  order  of  the  primary  part. 
Also  a  ranking  r  can  be  extended  to  a  ranking  {0, . . . ,  —  1}  of 

m-tuples.  On  the  other  hand,  a  ranking  need  not  be  first-order  definable  from  a 
linear  order;  take  e.g.  =  (N,  <).  However,  if  is  available  then  a  ranking  is 
definable  from  a  linear  order  by  r(a;)  =  x[y  <  «]. 

.  We  write  Tlr  for  the  class  of  ranked  arithmetical  structures  of  vocabulary  T. 
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Lemma  4.9  (Coding  Lemma.)  For  every  vocabulary  T  of  ranked  arithmetical 
structures  there  exists  an  encoding  function 

c:1lr—*n’= 

with  the  following  properties: 

(i)  c  is  definable  by  first-order  terms; 

(a)  The  primary  part  and  the  weight  functions  o/S)  can  be  reconstructured 
from  c(2))  in  polynomial  time; 

(Hi)  there  exists  a  polynomial  p(n,m)  such  that  c,(3))  <  for 

every  i<k. 

Proof  Encode  every  weight  function  w  :  — >■  N  by  a  pair  (q,  s)  of  natural 

numbers,  where 

q  —  +  1 

X 

s  = 

X 

This  encoding  is  first-order  definable:  for  q  this  is  obvious,  and 

s = '•-»(»)))  • 

X  y 

To  encode  2)  we  pass  to  the  associated  algebra  2)^  and  represent  it  by  the 
sequence  of  pairs  (g,s)  that  encode  the  weight  functions  of  2)“.  Obviously,  prop¬ 
erties  (i),  (a),  (Hi)  are  satisfied.  □ 

Theorem  4.10  Every  recursively  enumerable  class  of  arithmetical  structures  is 
a  metafinite  spectrum. 

Proof.  Let  K  C  Mr[9t]  be  recursively  enumerable.  Then  the  set 

ciJC)  :=  {c(2),  r)  :  2)  G  /C,  r  is  a  ranking  of  2)}  C 

is  also  recursively  enumerable  and  therefore  Diophantine.  The  desired  first-order 
sentence  ^  uses  besides  the  symbols  of  T  a  unary  weight  function  r  and  nullary 
weight  functions  and  expresses  (i)  that  r  is  a  ranking  and  (ii)  that 

Q(c(2),  r),  6))  =  0  for  a  suitable  polynomial  Q  €  Z[a:i, . . . ,  a:*,  ?/i, . . . ,  ym]  defin¬ 
ing  c(/C).  □ 

Conversely,  it  is  easy  to  see  that  every  metafinite  spectrum  of  arithmetical 
structures  is  recursively  enumerable,  so  we  obtain: 

Corollary  4.11  On  arithmetical  structures,  metafinite  spectra  capture  the  r.e. 
sets. 


339 


But  there  are  other  contexts  where  metafinite  spectra  do- indeed  capture  (a 
suitable  notion  of)  nondeterministic  polynomial- time.  An  important  example  are 
computations  over  the  real  numbers  with  the  model  of  Blum-Shub-Smale. 

Theorem  4.12  (Gradel,  Meer)  NPi  coincides  with  the  class  of  metafinite 
spectra  of'R-structures. 

The  proof  is  given  in  [16]. 

Definition  4.13  Let  JC  C  and  suppose  that  we  have  fixed  a  cost  func¬ 

tion  on  R.  We  say  that  /C  is  a  polynomially  bounded  metafinite  spectrum  if  there 
exists  a  first-order  sentence  of  vocabulary  T'  D  T  and  a  polynomial  p(n,  m) 
such  that  JC  is  the  class  of  all  2)  E  Mr[lH]  for  which  there  exists  an  expansion 
'D'  with 

-  2)'  1= 

-  .max2)'  <  p(|2)|,max2)) 

Remark.  If  the  cost  function  is  universally  bounded  by  a  constant  (as  in  the 
case  of  M-structures),  then  trivially  every  metafinite  spectrum  is  polynomially 
bounded. 

Conjecture  4.14  Let  JC  C  Mr[^]  be  a  class  of  arithmetical  structures,  which 
is  closed  under  isomorphism.  Then  the  following  are  equivalent: 

(i)  /C  E  NP. 

(a)  JC  is  a  polynomially  bounded  metafinite  spectrum. 

It  is  not  difficult  to  prove  that  every  polynomially  bounded  metafinite  spec¬ 
trum  is  in  NP,  i.e.  that  (ii)  implies  (i).  The  other  direction  is  related  to  a  conjec¬ 
ture  of  Adleman  and  Manders  concerning  the  notion  of  Diophantine  complexity 
(see  [3,  4,  26,  32,  36,  38]). 

Adleman  and  Manders  introduced  the  class  D  of  all  relations  S  C  that 
can  be  represented  in  the  form 

m 

a  G  5  3yi  •  •  •  3j/.„  (/\  y.-  <  2'"“‘  "“‘H'  A  Qia,  y)  =  o) 

for  some  ^  E  N  and  some  polynomial  Q  with  integer  coefficients.  They  conjec¬ 
tured  that  every  arithmetical  relation  in  NP  can  be  given  such  a  Diophantine 
representation,  i.e.  that  D  =  NP.  A  positive  solution  would  imply  (and  in  fact 
be  equivalent  to)  Conjecture  4.14. 

It  is  obvious  that  the  analogue  of  Conjecture  4.14  for  PTA-structures  is  true, 
since  there  we  have  all  polynomial-time  computable  functions  available.  But  in 
fact,  much  weaker  expansions  of  91o  will  do  as  well.  Let  Tl  be  obtained  from  91o 
by  adding  at  least  one  of  the  following  functions  or  relations: 
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-  the  so-called  logical  and  function,  mapping  numbers  a,  b  with  binary  expan¬ 
sions  a  —  ^  ==  Yli=o  to 

min(f,m) 

akb  ^  min(a,“,  6i)2*. 

»=o 

-  the  partial  order  X  with  a  6  iff  akb  =  a,  (i.e.  every  bit  of  a  is  less  than  or 

equal  to  the  corresponding  bit  of  b); 

-  the  function  (a,  6,  c) »— >■  (J)  (mod  c); 

-  the  modular  factorial  function  (a,  6)  a!  (mod  6). 

Then,  results  of  Jones  and  Matijasevich  [32]  imply 

Theorem  4.15  Every  class  in  NP  of  arithmetical  structures  with  secondary  part 

25  a  polynomially  bounded  metafinite  spectrum. 

The  ordering  •<  or  the  logical  and  can  be  directly  used  to  describe  compu¬ 
tations.  Binomial  coefficients,  and  therefore  factorials,  suffice  to  define  ^  since 
a  6  if  and  only  if  (j)  is  odd.  This  follows  from  Lucas’  theorem  that,  for  every 
prime  p,  given  p-ary  representations  a  =  ^  we  have  that 

(?)=n.a;)(modp). 

We  can  reformulate  Theorem  4.15  as  follows.  If  JC  is  an  isomorphism-closed 
class  of  arithmetical  structures  with  secondary  part  (or  PTA),  then  tC  is  in 
NP  if  and  only  if  it  can  be  characterized  as  the  model  class  of  a  second-order 
sentence  with  bounded  quantifiers  in  the  following  way: 

TieJC  iff  5)  1=  {3Fi  <  . . .  {3Fk  < 

where  ^  is  first-order  and  p  is  a  polynomial.  Here  {3Fi  <  2^^’^’"^)) ...  is  to  be 
understood  as  an  abbreviation  for  3jP,  [V^(F,(^)  <  2^(1^! A  . . .]. 

From  results  of  Hodgson  and  Kent  [26,  36],  we  obtain  a  more  involved  char¬ 
acterization  that  works  also  for  the  secondary  part  91,  and  in  fact  also  for  simple 
arithmetical  structures.  Here,  the  second-order  prefix  has  besides  the  exponen¬ 
tially  bounded  existential  quantifiers  (BF*  <  2^^”’”^^),  also  polynomially  bounded 
universal  quantifiers  of  the  form  (VG,-  <  p(n,  m)).  Hodgson  and  Kent  proved  that 
if  one  generalizes  the  class  D  of  Adleman  and  Manders  by  allowing  also  poly¬ 
nomially  bounded  universal  quantifiers  in  the  prefix,  then  one  obtains  a  precise 
arithmetical  characterization  of  NP.  In  fact  one  can  even  do  away  with  all  but 
one  of  these  universal  quantifiers  and  obtain  a  normal  form  which  is  the  analogue 
to  the  so-called  Davis  normal  form  for  r.e  sets.  The  Davis  normal  form  theorem 
says  that  every  recursively  enumerable  set  5  C  N*  can  be  represented  as 

5  =  {a  eN*  :  32/i(V«  <  yi)3j/2  =  0} 

(where  Q  €  S[2?i, . . . ,  2/1?  •  •  •  ^  2/m5  z])]  it  was  an  important  step  towards  the 

eventual  solution  of  Hilbert’s  10th  problem  by  Matijasevich.  For  NP-classes  of 
arithmetical  structures  this  gives  the  following  logical  characterization. 
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Theorem  4.16  An  isomorphism-closed  class  K  C  Mrpl]  is  in  NP  if  and  only 
if  there  exists  a  first-order  formula  xj)  and  a  polynomial  p{n,m)  such  that  JC  is 
the  class  of  all  2D  G  Mrpl]  with 

3?  1=  (3Fi  <  2P(”-’”))(VG  <  p(n,  m))(3jF’2  <  •  •  •  (3Fjt  < 

4.3  Fixed  point  logics  and  polynomial-time 

Fixed  point  logics  on  finite  structures.  In  finite  model  theory,  fixed  point 
logics  play  a  central  role.  They  provide  a  general  and  flexible  method  of  in¬ 
ductively  defining  new  predicates  and  thus  remedy  one  of  the  main  deficiencies 
(with  respect  to  expressiveness)  of  first  order  logic:  the  lack  of  a  mechanism  for 
unbounded  recursion  or  iteration. 

We  recall  the  definition  of  (inflationary)  fixed  point  logic.  Let  Ta  be  a  vo¬ 
cabulary,  R^Ta  an  r-ary  predicate  and  'ip{x)  a  formula  of  vocabulary  Ta  U  {R} 
with  free  variables  x  =  . . . ,  Then  ip  defines  for  every  finite  Ta -structure 

Ql  an  operator  — >■  P(A'*)  on  the  class  of  r-ary  relations  over  A  by 

:R^R\j{a:{%R)\=  ^(a)}. 

By  definition,  this  operator  is  inflationary,  i.e  R  C  F^{R)  for  all  R  C  A^ , 
Therefore  the  inductive  sequence  R^,R^,...  defined  by  :=  0  and  := 

F^{R^)  is  increasing,  i.e.  R^  C  R^"^^  and  therefore  reaches  a  fixed  point  R^  = 
R^'^^  for  some  j  <  \A\'^.  It  is  called  the  inflationary  fixed  point  of  ^  on  21,  and 
denoted  by  . 

Definition  4.17  The  (inflationary)  fixed  point  logic  FP  is  defined  by  adding  to 
the  syntax  of  first  order  logic  the  fixed  point  formation  rule:  if  ‘tp(x)  is  a  formula 
of  vocabulary  <7  u  {i?}  as  above  and  u  is  an  r-tuple  of  terms,  then 

is  a  formula  of  vocabulary  To,  whose  semantic  is  that  u  G  R°^. 

Example  4.18  Here  is  a  fixed  point  formula  that  defines  the  reflexive  and  tran¬ 
sitive  closure  of  the  binary  predicate  E: 

TC(w,  v)  =  [FPT,a:,y  (x  =  y)  W  {^z){Exz  A  Tzy)]{u,  v). 

Many  other  variants  of  fixed  point  logics  have  been  studied,  most  notably 
the  least  fixed  point  logic,  denoted  LFP,  and  the  partial  fixed  point  logic,  de¬ 
noted  PFP.  It  was  proved  independently  by  Immerman  [28]  and  Vardi  [48]  that, 
on  ordered  finite  structures,  LFP  characterizes  precisely  the  queries  that  are 
computable  in  polynomial  time.  Gurevich  and  Shelah  [23]  proved  that  FP  and 
LFP  have  the  same  expressive  power  on  finite  structure,  so  in  particular,  FP 
also  characterizes  Ptime  in  the  presence  of  a  linear  ordering.  On  the  class  of 
arbitrary  (not  necessarily  ordered)  finite  structures,  FP  and  LFP  are  strictly 
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weaker  than  PxiME-computability.  In  fact,  on  very  simple  classes  pf  structures, 
such  as  structures  with  the  empty  vocabulary  (i.e.  pure  sets),  FP  collapses  to 
first-order  logic.  Also,  the  0-1  law  holds  for  FP,  which  shows  that,  on  arbitrary 
finite  structures,  FP  cannot  express  non-trivial  statements  about  cardinalities. 

The  fixed  point  logic  FP*.  Definition  3.1  gives  a  general  way  of  extending  a 
logic  L  for  finite  structures  to  a  logic  L*  of  metafinite  structures.  Applying  this 
definition  to  FP,  we  get  the  logic  FP*,  the  extension  of  first-order  logic  FO*  by 
the  rule  for  building  fixed  point  formulae  [FP^,^  ^](w)  of  vocabulary  (T^,  Tr,  T^;) 
from  already  given  formulae  ^  of  vocabulary  (Tq  U  {i2},Tr,Tty).  It  is  important 
to  emphasize  that  the  inductively  defined  predicate  is  a  predicate  over  the 
primary  part  and  that  u  is  a  tuple  of  point  terms.  We  first  observe  that  the  fixed 
point  construction  preserves  PTiME-comput ability. 

Proposition  4.19  // f/ie  basic  functions,  relations  and  multiset  operations  of 
91  can  he  evaluated  in  polynomial  time  (with  respect  to  the  given  cost  function), 
then  the  same  is  true  for  all  YF* -definable  global  function  on  Mr  [91]. 

As  in  the  case  of  Fagin’s  Theorem  we  can  also  transfer  Immerman’s  and 
Vardi’s  logical  characterization  of  Ptime  to  the  case  of  arithmetical  structures 
with  small  weights. 

Theorem  4.20  Let  JC  C  Hr  a  class  of  ranked  arithmetical  structures  with 
small  weights.  For  every  global  function  G  on  JC  the  following  are  equivalent 

(i)  G  is  computable  in  polynomial  time. 

(ii)  G  is  YF* -definable. 

We  omit  the  proof,  which  follows  by  straightforward  application  of  the  same 
arguments  as  in  the  proof  of  Theorem  4.6. 

Again,  as  in  the  case  of  metafinite  spectra,  the  restriction  to  small  weights  is 
necessary.  For  an  extreme  example,  consider  polynomial-time  predicates  5  C  N. 
Each  such  S  gives  rise  to  a  decision  problem  where  an  instance  is  an  arithmetical 
structure  2),  with  a  single  nullary  weight  a,  and  the  question  is  whether  a  G  5. 
Of  course  this  problem  is  completely  independent  of  the  primary  part  of  the 
structure,  which  in  particular  can  be  trivial.  Fixed  point  constructions  are  of 
absolutely  no  help  here  and  neither  are  quantifiers  or  multiset  operations.  Thus 
FP*  can  decide  S  if  and  only  if  the  characteristic  function  X5(a)  is  available  as 
a  basic  term.  Obviously  there  exist  polynomial- time  predicates  S  for  which  this 
is  not  the  case. 

Thus,  FP*  cannot  fully  capture  Ptime  on  arithmetical  structures,  even  in 
the  presence  of  a  ranking. 

But  this  is  not  the  only  weakness  of  FP*.  Another  important  limitation  is  the 
absence  of  any  recursion  mechanism  over  numbers  and  weight  functions.  We  will 
exhibit  certain  interesting  consequences  of  this,  by  comparing  the  power  of  FP* 
with  the  fixed  point  logic  with  counting  (FP  +  C)  on  unordered  structures.  This 
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logic  does  not  include  large  numbers  in -the  secondary  sort,  but  it  has  recursion 
over  relations  that  range  over  both  parts. 

Fixed  point  logic  vv^ith  counting.  As  we  mentioned  already  in  the  introduc¬ 
tion,  among  the  logics  studied  in  finite  model  theory,  (FP  -f  C)  is  the  closest  to 
our  approach.  It  was  first  proposed  by  Immerman,  who  started  from  the  obser¬ 
vation  that  counting  is  probably  the  most  basic  class  of  low-complexity  queries 
not  expressible  in  fixed  point  logic.  The  original  hope  was  that  the  addition  of 
counting  to  FP  in  a  reasonable  way  should  give  a  logic  that  could  express  all  of 
Ptime.  It  should  be  pointed  out,  that  there  are  different  ways  of  adding  counting 
mechanisms  to  a  logic,  which  are  not  necessarily  equivalent.  The  most  straight¬ 
forward  possibility  is  the  addition  of  quantifiers  of  the  form  3-^,  3-^,  etc.,  with 
the  obvious  meaning.  While  this  is  perfectly  reasonable  for  the  infinitary  logics 
j  if  i*^  general  enough  for  fixed  point  logic,  because  it  does  not  allow  to 
apply  recursion  also  on  the  counting  parameters  i  in  quantifiers  3-* a:.  In  fact  if 
the  counting  parameters  are  fixed  numbers,  then  adjoining  the  quantifiers  3-*  a; 
does  not  give  additional  power  to  logics  whose  formulae  may  have  an  arbitrary 
number  of  variables  (as  FO  or  FP).  These  counting  parameters  should  therefote 
be  considered  as  variables  that  range  over  the  natural  numbers.  To  define  in 
a  precise  way  a  logic  with  counting,  and  with  recursion  applicable  also  to  the 
numbers  obtained  by  counting,  it  is  therefore  necessary  to  extend  the  original 
objects  of  study,  namely  finite  (one-sorted)  structures  21  to  two-sorted  auxiliary 
structures  21*  with  a  second  numerical  (but  also  finite)  sort. 

We  are  now  ready  to  formally  introduce  (FP  -f-  C).  With  any  one-sorted  finite 
structure  2t  one  associates  the  two-sorted  structure  2t*  :=  (21,  (n,  <))  consisting 
of  a  copy  of  21  for  the  first  sort  and  the  linear  order  (n,  <)  for  the  second  sort, 
with  n  =  \A\-{-l  and  the  standard  meaning  of  <  on  n  =  {0, . . . ,  n  —  1}.  We  take 
n  =  |A|  +  1  rather  than  n  =  \A\  to  be  able  to  represent  the  cardinalities  of  all 
subsets  of  |A|  within  n. 

We  start  with  first-order  logic  over  two-sorted  vocabularies  (Ta,{<}),  with 
the  usual  semantics  over  structures  21*.  Latin  letters  x^y^z,. . .  are  used  for 'the 
variables  over  the  first  sort,  and  Greek  letters  A,  //,  i/, . . .  for  variables  over  the 
second  sort.  Note  that,  contrary  to  logics  of  metafinite  structures,  we  have  here 
no  restriction  on  the  access  of  the  logic  to  second  sort  elements.  For  instance, 
we  can  quantify  over  number  variables  to  build  formulae  of  the  form  3fji(p. 

The  two  sorts  are  related  by  counting  terms,  defined  by  the  following  rule:  Let 
(p{x)  be  a  formula  with  a  free  variable  x  of  sort  one,  then  is  a  second-sort 

term,  with  the  set  of  free  variables  free(#a;[y?])  =  free(^)  —  {x}.  The  interpreta¬ 
tion  of  is  the  number  of  first-sort  elements  a  that  satisfy  (p{a).  First-order 

logic  with  counting,  denoted  (FO  -f  C),  is  the  closure  of  two-sorted  first-order 
logic  under  counting  terms. 

Example  4.21  To  illustrates  the  use  of  counting  terms  we  present  a  formula 
xp{Ei ,  E2)  6  (FO  -f  C)  expressing  that  two  equivalence  relations  Ei  and  E2  over 
the  first  sort  are  isomorphic. 

HEuE^)  :=  (y/i)(#.[#y[£iX3/]  =  ^l]  =  #.[#,[£2*2/]  =  /*]). 
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The  (inflationary)  fixed  point  logic  with  counting  (FP  +  C)  is  obtained  by 
adding  to  (FO  +  C)  the  mechanism  for  building  fixed  point  predicates  that  may 
range  over  both  sorts. 

Definition  4.22  The  logic  (FP  +  C),  is  the  closure  of  two-sorted  first-order 
logic  under 

(i)  the  rule  for  building  counting  terms; 

(a)  the  usual  rules  of  first-order  logic  for  building  terms  and  formulae; 

(Hi)  the  fixpoint  formation  rule:  Suppose  that  is  a  formula  of  vo¬ 
cabulary  T  U  {i^}  where  x  =  xi,. .  .,Xk,  =  pi, . . .  and  R  has  mixed 

arity  and  that  (u,i>)  is  a  A:  -f  ^-tuple  of  first-  and  second-sort  terms, 

respectively.  Then 

is  a  formula  of  vocabulary  T. 

The  semantics  of  [FP/^^^^^  on  21*  is  defined  in  the  same  way  as  for  the 
logic  FP,  namely  as  the  inflationary  fixed  point  of  the  operator 

Ff  :  i?  ^  «  U  {(«,  P)  I  (a* ,  i?)  [=  V(«,  P)}. 

(FP  -f  C)  was  first  introduced  by  Immerman,  in  a  different  but  equivalent 
form,  with  counting  quantifiers  rather  than  counting  terms.  The  present  version 
appeared  first  in  [17]. 

Example  4.23  An  interesting  example  for  an  (FP  -f  C)-computable  global 
function  is  the  stable  colouring  of  a  graph.  Given  a  graph  G  with  a  colour¬ 
ing  f  :V  — ^  0, . . . ,  r  of  its  vertices,  we  define  a  refinement  f  of  /,  where  vertex 
X  has  the  new  colour  fx  =  {fx,  ni , . . . ,  where  n*  =  #y[Eary  A  (/y  =  i)].  The 
new  colours  can  be  sorted  lexicographically  so  that  they  form  again  an  initial 
subset  of  N.  Then  the  process  can  be  iterated  until  a  fixed  point,  the  stable 
colouring  of  G  is  reached.  It  is  known  that  almost  all  graphs  have  the  property 
that  no  two  vertices  have  the  same  stable  colour.  Thus  stable  colourings  provide 
a  polynomial-time  graph-canonization  algorithm  for  a  dense  class  of  graphs.  It 
should  be  clear  that  the  stable  colouring  of  a  graph  is  definable  in  (FP  -|-  C)  (see 
[31]  for  more  details). 

Over  arithmetical  structures,  we  can  define  counting  in  FO*  and  hence  FP*, 
as  shown  in  Example  3.3.  One  might  therefore  feel  that  FP*,  having  both  a  fixed 
point  constructor  and  the  ability  to  count,  is  at  least  as  powerful  as  (FP  -h  C). 

To  make  this  a  precise  question,  we  have  to  consider  a  setting  where  the 
two  logics  can  be  compared.  We  compare  their  expressive  powers  on  classes 
K  C  Fin(Ta)  of  finite,  one-sorted  structures. 

Definition  4.24  With  every  finite  structure  21  and  every  secondary  part 
we  associate  the  metafinite  structure  21jh  :=  (21,91,0),  with  primary  part  21, 
secondary  part  91  and  the  empty  set  of  weight  functions.  We  say,  that  a  model 
class  K  C  Fin(Ta)  of  finite  structures  is  FP*-definable  over  91,  if  there  exists  a 
sentence  ^  G  FP*  such  that 
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/C  =  {21  6  Fin(ra)  :  21,^  (=  V-}. 

As  usual  we  say  that  JC  is  (FP  +  C)-definable  if  there  exists  a  sentence  9  € 
(FP  +  C)  such  that 

/C  =  {21  G  Fin(ra)  :  2t*  ^  9}. 

Proposition  4.25  Let  01  he  any  reduci  of  PTA.  Then  every  model  class  1C  C 
Fin(Ta)  which  is  FT* -definable  over^,  is  also  (FP  +  C)-definable. 

Proof.  This  follows  by  straightforward  induction  over  terms  and  formulae  of  FP* , 
using  the  facts  that  (i)  every  FP*-definable  global  function  can  be  evaluated  in 
polynomial  time  and  that  (ii)  every  polynomial-time  computable  function  or 
relation  appearing  in  the  secondary  part  can  be  expressed  by  an  (FP  -1-  C)- 
definition  over  the  numerical  sort  (since  the  numerical  sort  is  ordered).  □ 

The  converse  is  not  always  true.  Indeed,  let  Ot  =  Olo*  If  we  consider  the  case 
that  Ta  =  0,  then,  by  taking  cardinalities,  a  class  JC  C  Fin(0)  can  be  viewed  as 
a  set  of  natural  numbers.  On  Fin(0),  (FP  -b  C)  captures  polynomial- time  with 
respect  to  the  cardinality  of  the  structures,  i.e.  K  C  Fin(0)  is  (FP  -|-  C)-definable 
if  and  only  if  {!”  :  n  G  /C}  is  decidable  in  polynomial  time.  On  the  other  hand, 
FP*  on  structures  (A,^0}  0)  is  equivalent  to  FO*  whose  power  can  be  precisely 
described  as  follows:  Every  sentence  (p  can  be  written  as  a  Boolean  combination 
of  inequalities  /(n)  <  g(n)  where  f,g  €  T  are  terms  in  one  variable  n  that 
represents  the  cardinality  of  A.  Since  all  elements  of  A  are  indistinguishable,  the 
terms  F  or  F  produced  by  means  of  the  multiset  operations  can  simple 
be  rewritten  as  n  •  F  and  F”  respectively.  (Applications  of  max  and  min  have 
no  effect  at  all.)  Thus  the  set  T  of  terms  can  be  defined  by  closing  the  constants 
and  n  under  addition,  multiplication  and  under  raising  to  nth  power  (i.e.  given 
t{n),  one  can  form  t(n)”).  A  simple  diagonalization  arguments  proves  that  there 
exist  predicates  5  C  N  which  cannot  be  defined  in  this  way,  but  nevertheless 
{!”  :  n  G  *9}  is  decidable  in  polynomial  time. 

Indeed,  let  ^  be  a  Boolean  combination  of  inequalities  f  <  g  with  f,g  €  T. 
Syntactically,  ^  is  a  string  in  a  finite  alphabet  whose  symbols  are  0,  l,n,-|-,  •, 
etc.  We  can  order  this  alphabet  and  assign  numbers  to  strings  in  the  usual  way. 
Let  n(ip)  be  the  number  associated  with  (p  and  S  be  the  set  of  those  numbers 
n{<p)  such  that  <p  is  false  at  n{(p).  Clearly,  S  is  not  defined  by  any  p.  Moreover, 
since  p  is  equivalent  to  (^  A 0  <  1),  {pAQ<  1  A 0  <  1)  etc.,  p  differs  from  S  on 
infinitely  many  numbers. 

It  thus  suffices  to  prove  that  there  exists  a  polynomial-time  algorithm  that, 
given  computes  the  truth  value  of  p  at  n{p)  and  inverts  the  result.  This 

is  obvious,  once  we  have  checked,  by  an  easy  induction  on  the  formation  rules 
of  T,  that  for  every  term  /  G  T,  the  logarithm  of  the  value  /(n)  is  bounded  by 
a  polynomial  in  n. 

We  thus  have  proved  the  following  result. 

Proposition  4.26  There  exist  model  classes  K  of  finite  structures  which  are 
(FP  -b  C)-definable,  but  not  FF* -definable  overdo. 
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The  fact  that  91o  forms  a  counterexample  to  the  converse  of  Proposition  4.25 
survives  various  enrichments  of  In  fact,  the  same  proof  works  if  is  ex¬ 
tended  by  any  finite  collection  of  polynomial-time  computable  functions  and  any 
finite  collection  of  multiset  operations  F  such  that  the  value  of  F  at  multisets 
consisting  of  n  occurrences  of  i,  can  be  computed  in  polynomial¬ 
time  with  respect  to  n  and  logt.  However,  there  is  a  limit  to  such  generalizations. 
We  will  prove  in  Sect.  5  that  the  converse  of  Proposition  4.25  does  hold' in  the 
case  that  91  =  PTA. 

Remark.  Note  that  the  problem  of  capturing  polynomial-time  on  ranked  PTA- 
structures  is  trivial  and  does  not  require  a  fixed-point  construction.  As  pointed 
out  above,  if  a  ranking  is  available,  then  the  primary  part  can  be  encoded  by 
a  tuple  of  natural  numbers  and  this  encoding  is  definable  by  first-order  terms. 
Any  polynomial-time  property  is  thus  reducible  to  a  Ptime  property  of  num¬ 
bers  which  is  a  basic  relation  of  PTA.  Thus  a  global  function  on  ranked  PTA- 
structures  is  PTIME-computable  if  and  only  if  it  is  first-order  definable.  Further¬ 
more  FO*  and  FP*  coincide  on  ranked  PTA-structures. 


4.4  A  functional  fixed  point  logic 

One  possibility  to  overcome  the  limitations  of  languages  of  type  L*  is  to  apply 
recursion  in  one  way  or  another  to  weight  functions. 

We  discuss  here,  as  one  particular  example,  &  fixed-point  calculus  ioi  partially 
defined  weight  functions.  It  is  convenient  to  deal  with  partial  functions  by  ex¬ 
tending  the  secondary  part  by  a  new  element  to  a  structure  fH*  with  universe 
R  U  {undef}  in  the  following  way: 

The  relations  of  91*  coincide  with  their  restrictions  to  91  and  the  functions 
and  multiset  operations  of  91  are  extended  to  91*  in  some  arbitrary  way.  For 
many  functions,  the  natural  choice  will  be  to  set  (a)  =  undef  whenever 
the  argument  a  contains  undef.  However,  for  some  functions  there  are  other 
reasonable  possibilities:  For  multiplication,  it  actually  makes  more  sense  to  set 

a  •  undef  =  undef  -  0  =  1^ 

undef  if  a  0. 

Fix  a  signature  T  and  a  function  symbol  Z  not  contained  in  T.  Let  G{Z,  x) 
be  a  weight  term  of  signature  (Ta  {Z})  and  free  variables  x  =  xi,. .  .,Xr 

where  r  is  the  arity  of  Z.  We  write  for  the  value  of  G(Z,  x)  for  a  given 

interpretation  ('S),Z). 

For  every  structure  3!)  €  the  term  G{Z^x)  gives  rise  to  an  operator 

Fq  which  updates  partially  defined  functions  Z  as  follows: 


if  Z{a)  —  undef 
otherwise. 


This  gives  an  inductive  definition  of  a  sequence  of  partial  weight  functions 


Z^  is  undefined  everywhere  (i.e.  Z^{a)  =  undef  for  all  a) 

=F§{Z^). 
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The  operator  Fq  updates  Z  only  at  points  where  Z  is  •  undefined,  so  this 
process  reaches  a  fixed  point  after  a  polynomial  number  of  iterations:  Z^  =  Z^"^^ 
for  some  j  <  We  denote  this  fixed  point  by  Z°°  and  call  it  the  fixed  point 
of  G{Zy  x)  on  25. 

Definition  4.27  Functional  fixed  point  logic^  denoted  FFP,  is  obtained  by  aug¬ 
menting  the  first-order  term  calculus  FOT  (see  Definition  3,8)  with  the  following 
rule  for  building  terms: 

If  G{Z,  x)  is  a  weight  term  of  signature  (TajTr^T^j  U  {^}),  if  x  =  xi, . . . , 
is  a  tuple  of  variables  (where  r  is  the  arity  of  Z),  and  if  n  =  ui, . . . ,  is  a  tuple 
of  point  terms,  then 

fp[Z{x)^G{Z,x)]{u) 

is  a  weight  term  of  signature  (Ta,  Tr,  Tu^).  Its  semantic,  on  a  given  structure  25, 
is  Z^(u). 

Note,  that  on  arithmetical  structures,  FFP  can  define  weights  of  double  ex¬ 
ponential  size.  Indeed  suppose  we  have  arithmetical  structures  with  a  ranking  r 
and  let  us  adopt  the  conventions  that  max  and  -|-  produce  undef  whenever  any 
of  the  arguments  is  undefined,  and  that  0  ♦  undef  =  0.  Set 

G{Z,  x)  :=  2x[r(j:)  =  0]  +  max(x[r(a:)  =  r{y)  +  !]]][  ^(v))  ■ 

Z 

Then,  for  every  structure  2)  with  |25|  =  n  we  have  that 

fp[Z(x)^G{Z,x)]iy)  =  2'^^^''\ 

This  even  works  for  simple  arithmetical  structures,  because  the  term  Z[y) 
—  which  evaluates  to  Z{yY‘  —  can  be  simulated  by  a  fixed  point  construction. 

However,  in  the  context  of  computations  over  M  with  the  Blum-Shub-Smale 
model,  the  magnitude  of  the  numbers  is  no  serious  problem,  since  one  assumes 
unit  cost  for  each  r  G  M.  In  fact  it  has  been  shown  in  [16]  that  functional 
fixed-point  logic  is  the  right  logic  for  describing  polynomial- time  computability 
in  that  model,  in  the  sense  that  it  gives  rise  to  the  following  analogue  of  the 
Immerman-Vardi  Theorem. 

Theorem  4,28  (Gradel,  Meer)  On  ranked  R-structures,  FFP  captures  Pi. 


Remark.  For  some  applications  the  update  operator  F^  used  for  FFP  may  not 
seem  adequate,  since  the  values  different  from  undef  are  never  updated.  Instead 
we  may  consider  a  different  update  operator  Fq  with 

F^{Z){a)-.=  G^'^{a). 
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Of  course,  the  inductive  process  defined  by  such  an  operator  need  not  reach  a 
fixed  point.  But  —  as  in  the  case  of  the  partial  fixed  point  logic  PFP  considered 
in  finite  model  theory  —  we  can  define  to  be  the  fixed  point  of  the  sequence 
defined  by  ,  if  the  fixed  point  exists,  and  some  default  value,  e.g. 
the  everywhere  undefined  function,  otherwise. 

We  don’t  further  investigate  this  approach  here.  The  study  of  this,  and  related 
variants  of  functional  fixed  point  logics,  as  well  as  other  means  of  inductive 
definability  of  weight  functions,  is  one  of  the  promising  directions  for  future 
research. 

5  Back  and  forth  from  finite  to  metafinite  structures 

As  explained  in  the  introduction,  our  goal  is  to  extend  the  approach  and  methods 
of  finite  model  theory  to  the  more  general  class  of  metafinite  structures.  We  show 
in  this  section  that  an  important  methodology  of  finite  model  theory,  namely 
the  various  variants  of  Ehrenfeucht-Fraisse  games,  is  indeed  applicable  in  our 
more  general  context. 

The  aspect  that  we  consider  here  is  the  indistinguishability  of  two  metafi¬ 
nite  structures  by  (infinitary)  logics  with  bounded  variables,  but  with  arbitrary 
multiset  operations.  We  show  that  this  reduces  to  the  indistinguishability  of  two 
associated  finite  structures  by  order  formulae  with  counting. 

Throughout  this  section,  we  consider  structures  with  a  fixed  secondary  part 
and  assume  that  the  primary  part  is  always  relational. 


5,1  Indistinguishability  by  logics  with  k  variables 

Definition  5.1  Let  3!)  =  (21,  SH,  W)  and  D*  —  (25,91,14^')  be  structures  in 
Mr  (91),  let  d  and  6  be  ^-tuples  of  elements  of  21  and  ©  respectively,  and  let  L  be 
a  logic  of  metafinite  structures.  We  say  that  (3),d)  and  (S',  6)  are  X-equivalent 
—  in  symbols:  (S,  a)  =l  (S',  6)  —  if  for  every  weight  term  F{xi, . . . ,  of  L, 

F'^{a)  =  F®'(6). 

Since  in  our  logics  we  have  for  every  formula  its  characteristic  function  avail¬ 
able  as  a  weight  term,  the  L-equivalence  of  (S,  d)  and  (S',  b)  implies  in  particular 
that  for  every  formula  (p{x)  of  L 

S  1=  (p{a)  if  and  only  if  S'  \=  <p{b). 

The  converse  does  not  necessarily  hold,  i.e.,  two  structures  may  be  indistin¬ 
guishable  by  formulae  of  L  but  there  nevertheless  may  exist  a  weight  term  that 
separates  them.  This  may  be  the  case  when  91  contains  unreachable  elements 
which  do  not  appear  as  values  of  any  closed  T,.-term. 

Logics  with  k  variables.  We  first  recall  the  definitions  of  some  logics  with 
bounded  number  of  variables  that  are  of  great  importance  in  finite  model  theory. 
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is  the  fragment  of  first-order  logic  consisting  of  the  formulae  whose  vari¬ 
ables,  both  free  and  bound,  are  among  jri, . . . ,  Xk.  The  infinitary  logic  is 
the  closure  of  under  conjunctions  and  disjunctions  applied  to  arbitrary  sets 
of  formulae.  Further,  =  IJfcew  i®  well-known  that  the  familiar  fixed 

point  logics  LFP,  IFF  and  PFP  are  sublogics  of  . 

The  logics  and  are  the  extension  of  L*,  and  by 

means  of  counting  quantifiers  3-^,  3-^,  etc.,  with  the  obvious  semantic.  One  of 
the  recLSons  why  these  logics  are  important  is  that  is  an  extension  of  fixed 
point  logic  with  counting  (FP  +  C). 

Equivalence  with  respect  to  has  an  elegant  characterization  in  terms 
of  the  A:-pebble  game  [6,  27,  43],  an  infinitary  variant  of  Ehrenfeucht-Fraisse 
games.  There  is  a  similar  pebble  game  appropriate  to  [31].  It  is  played  by 
two  players,  I  and  11,  on  two  structures  21  and  fB  of  the  same  relational  signature. 
They  have  k  pairs  of  pebbles. 

A  move  of  the  game  is  played  as  follows. 

\ 

1.  Player  I  chooses  i  <  k  and  picks  up  the  i-th  pair  of  pebbles.  He  selects  a 
nonempty  subset  X  of  either  A  or  B.  Player  II  chooses  a  subset  Y  in  the 
other  structure  with  \Y\  =  \X\,  If  no  such  set  exists,  the  game  is  over  and 
Player  I  has  won. 

2.  Player  I  places  an  i-pebble  on  an  element  y  €  Y.  Player  II  puts  the  other 
i-pebble  on  an  element  x  E  X. 

After  any  move,  the  pebbles  on  the  ‘board’  define  a  partial  map  from  A  to  jB, 
taking  every  pebbled  element  of  A  to  the  element  of  B  carrying  the  corresponding 
pebble.  Player  II  has  to  maintain  the  condition  that  the  pebble  map  is  a  partial 
isomorphism.  We  say  that  Player  II  wins  the  C^-game  on  (21,  ai, . . . ,  a^)  and 
(Q5,6i, . . . ,  6^)  if  she  has  a  strategy  to  maintain  this  condition  forever,  when 
initially  the  first  i  pairs  of  pebbles  are  placed  on  (fli,  6i), . . . ,  (at,  bt). 

Theorem  5.2  (Immerman,  Lander)  The  following  are  equivalent 

(i)  Player  II  wins  the  C^-game  on  (21,  d)  and  (03,6). 

(ii)  21 1=  (p{a)  iff^\=  (p(b)  for  every  formula  (p{x)  £ 

Here  is  another  way  to  put  and  to  refine  this  (see  [17,  41]).  For  a  tuple 
d  G  (A  U  {*})*'  (where  ♦  serves  as  a  dummy  value  in  the  case  that  not  all  k 
variables  are  actually  used)  we  write  dj  for  the  tuple  obtained  by  substituting 
(or  adding)  c  at  position  j  to  a. 

We  write  (2t,  d)  (03,6)  if  Player  II  has  a  strategy  to  maintain  the  win¬ 
ning  condition  for  at  least  i  moves  of  the  C^-game,  starting  at  position  (21,  d) 
and  (03,6).  Note  that  (21,  d)  ~o  (®j^)  if  sind  only  if  p  :  d  i — >■  6  is  a  partial 
isomorphism  from  21  to  03. 

Theorem  5.3  (21,  d)  (03,6)  if  and  only  2/(21,  d)  (05,6),  and  for  every 
^i- equivalence  class  C  and  every  j  <k  we  have  that 

#{c  €  A  ;  i%af)  e  C}  =  #{d  6  B  :  €  C}. 
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Since  -equivalence  is  the  intersection  of  all  equivalence  relations  one 
obtains  the  following  characterization. 

Theorem  5.4  equivalence  is  the  coarsest  equivalence  relation  ~  with  the 
following  properties  7/(21,  a)  ^  then 

(i)  The  function  p  :  a  i — >■  b  is  a  partial  isomorphism  from  21  to  05; 

(ii)  for  every  ^-equivalence  class  C  and  every  j  <  k  we  have  that 

#{c  G  A  :  (2t, af)  eC}  =  #{deB:  (!B, bf)  €  C}. 

An  infinitary  ^-variable  term  calculus  for  metafinite  structures  with 
multiset  operations.  Lqooj  generalizes  first-order  logic.  In  a  similar  vein,  we 
generalize  the  first-order  term  calculus  FOT(r)  given  by  Definition  3.8  and  the 
subsequent  remark  (because  Ta  is  not  necessary  empty).  Let  FOT^(T)  be  the 
restriction  of  FOT(T)  to  terms  using  only  the  variables  aii, . . . ,  Xk. 

•Define  a  set  operation  on  a  set  7J  to  be  a  unary  operation  from  subsets  of  R 
to  R.  Let  T*  be  the  extension  of  T  with  names  for  all  multiset  operations  over 
R,  and  let  fH*  be  the  corresponding  expansion  of  Vi. 

Definition  5.5  The  term  calculus  R)  is  the  extension  of  the  term  cal¬ 

culus  FOT*(r*)  with  the  following  rule:  If  5  is  a  set  operation  on  R  and  is  a 
set  (any  set)  of  terms,  then  S{^)  is  a  term.  The  rank  of  S{^)  is  the  supremum 
of  the  ranks  of  terms  in  ^  (which  may  be  an  infinite  ordinal).  The  semantics  is 
as  follows:  Given  an  evaluation  of  the  variables,  compute  the  set  X  C  R  of  the 
values  of  terms  in  ^  under  that  evaluation,  and  then  apply  S  to  X. 


Remark.  The  relation  of  F  being  a  proper  subterm  of  a  term  G  is  well  founded. 

Remark.  Let  us  see  that  the  characteristic  function  of  every  formula  (p 
about  the  primary  part  is  given  by  some  term  t^  in  The  characteristic 

functions  of  the  primary  relations  are  always  available.  If  ^  =  -i^  then  the  desired 
I'ip  =  S{{t^jj})  where  S  is  any  set  operation  such  that  5({0})  =  1  and  5({1})  =  0. 
If  ^  is  a  disjunction  of  formulas  v?,-  where  i  G  I  then  =  S{{t^i  :  i  G  7})  where 
S  is  any  operation  that  coincides  with  max  on  nonempty  subsets  of  {0,1}.  To 
handle  counting  quantifiers,  let  T*  be  a  multiset  operation  such  that  r*(m)  =  1 
if  m  contains  at  least  i  occurrences  of  1  aind  r*(m)  =  0  otherwise.  If  ^  =  3^^x(p 
then  tg,  = 

Example  5.6  Suppose  that  a  metafinite  structure  V  =  (21,  %  W)  is  such  that 
every  element  a  G  A  is  definable  in  21  by  some  formula  (pa{x),  and  W  contains 
a  unary  weight  function  w.  Let  5  be  a  set  function  such  that  S{X)  =  1  if  and 
only  if  every  number  in  X  is  prime.  The  the  term 

'5'({x[v^a](ic)  •  w(x)  :  o  G  A}) 

evaluates  to  1  in  ®  if  and  only  if  the  range  of  w  consists  of  primes. 
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Remark.  In  the  remainder  of  this  section,  we  prove  various  theorems  about  the 
term  calculus  The  developed  theory  is  quite  robust  with  respect  to  the 

definition  of  It  does  not  change  if  the  is  further  enriched  by  means 
of  even  fancier  super-operations  over  R;  for  example  we  may  require  that,  for 
every  finitary  or  infinitary  operation  /(ri,  r2j . . .)  over  R  and  terms  ti  €  the 
possibly  infinitary  expression  . . .)  is  a  term  in  On  the  other  hand, 

as  the  remark  above  shows,  we  actually  use  only  very  simple  set  operations. 


5.2  Partial  isomorphisms  and  the  multiset  pebble  game. 

Consider  a  metafinite  structure  2)  =  (21,  IH,  W)  G  Mr  (9^).  We  associate  with  2) 
a  finite  structure  fin(2))  with  universe  A,  by  expanding  21  with  relations 

Pw,r  :=  {d  :  u;^(d)  =  r} 

for  every  function  w  E  W  and  every  element  r  £  R.  Although  the  set  of  these 
new  predicates  is  infinite,  only  finitely  many  relations  are  nonempty. 

Definition  5.7  Let  2)  =  (21,91,  W)  and  25'  =  («,9l,  W')  belong  to  Mr (91).  A 
‘partial  isomorphism  from  2)  to  2)^  is  a  function  p  :  Aq  — >•  B  whose  domain  is 
Ao  C  A  such  that 

—  for  every  relation  symbol  R£Ta  and  all  elements  ai, . . . ,  6  Aq 

2)  \=  R{ai,, . . ,  Om)  if  and  only  if  2)'  |=  R{pai,. . .  ,pam)« 

—  for  every  function  symbol  w  £Txd  and  all  elements  ai, . . . , am  €  Aq  we  have 
that 

=w'^'(pai,...,pam)- 

Thus,  the  partial  isomorphisms  from  2)  to  2)'  are  precisely  the  partial  iso¬ 
morphisms  from  fin (2))  to  fin(2)'). 


We  now  describe  the  ‘obvious’  pebble  game  appropriate  to  the  logic 
Given  two  metafinite  structures  2)  =  (21, 91,  W)  and  2)'  =  (25, 91,  W')  in  Mr  (91), 
the  T^-game  on  (2),  2)')  is  played  with  k  pairs  of  pebbles  on  the  ‘board’  (A,  B). 
A  move  of  the  T*-game  is  played  as  follows: 

1.  Player  I  selects  £  <  k  pairs  of  pebbles  and  selects  a  function  f  :  A^  R. 
Player  II  chooses  a  function  g  :  B^  R  such  that  mult(/)  =  mult(^).  (Recall 
that  mult(/)  =  if  (a)  :  a  €  A^J.)  If  no  such  function  exists,  the  game  is 
over  and  Player  I  has  won. 

2.  Player  I  puts  the  selected  pebbles  on  elements  6i, . . . ,  G  jB.  Player  II  puts 

the  corresponding  pebbles  on  ai , . . . ,  such  that  /(oi , . . . ,  )  =  g{bi , . . . ,  6^) 
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Remark.  It  might  seem  that  there  is  an  asymmetry  here,  since  Player  I  always 
selects  a  function  on  the  first  structure  and  always  pebbles  elements  on  the 
second  one,  and  that  instead,  he  should  be  allowed  to  choose  on  which  structure 
he  defines  a  function.  However,  this  would  not  change  the  game  in  an  essential 
way.  The  condition  that  Player  II  answers  with  a  function  defining  the  same 
multiset  is  very  restrictive  and  makes  it  unnecessary  to  let  Player  I  choose  the 
structure  first.  In  particular,  I  wins  immediately  if  the  primary  parts  of  the 
two  structures  do  not  have  the  same  cardinality.  It  should  be  noted  that  if  two 
structures  2t  and  05  are  known  to  have  the  same  number  of  elements,  then  also 
the  C*-game  on  Ql  and  05  can  be  restricted  such  that  Player  I  always  chooses 
his  sets  in  and  pebbles  elements  of  fB,  but  never  vice  versa. 

The  moves  in  the  T^-game  simulate  the  use  of  the  multiset  operations.  How¬ 
ever,  it  turns  out  that  the  T^-game  is  equivalent  to  the  (7*-game  of  Immerman 
and  Lander.  We  prove  this  by  way  of  two  Lemmata. 

Lemma  5.8  Lei  F(x)  he  a  weighi  term  in  T*  of  rank  a  such  that  F'^(d)  4 
F'^\h).  Then 

(i)  Player  I  wins  the  C^-game  on  (fin(iD),a)  and  (fin(2)0,6).  Furthermore  if 

a  is  finite  then  he  wins  the  game  in  a  moves. 

(ii)  Player  I  wins  the  T^-game  on  (X>,a)  and  Furthermore  if  a  is 

finite  then  he  wins  the  game  in  a  moves. 

Proof  Obviously,  (i)  implies  (ii)._We  prove  (i)  by  induction  on  a,  the  case  a  =  0 
being  trivial.  Let  jP®(d)  ^  F'^'{b)  for  some  term  F  of  rank  »_>  0.  If  =  S(^) 
for  some  operation  S  and  set  of  terms  then  G'^{d)  ^  G®'(6)  for  at  least  one 
G  G  similarly,  if  F  =  g{Fi, . . . ,  F^)  then  at  least  one  subterm  F,-  separates 
(iD,d)  and  (2)',  6). 

Since  the  process  of  descending  to  proper  subterms  is  well-founded,  F  con¬ 
tains  at  least  one  subterm  separating  (£>,a)  and  (5)',  6)  which  either  is  of  rank 
zero,  in  which  case  we  are  done,  or  of  the  form 

ry(G{x,y):H{x,y)  =  l) 

where  G  and  H  have  ranks  <  a.  For  ease  of  notation,  we  assume  that  x  and 
y  are  disjoint  tuples  of  variables  among  a;i, . . . ,  a;*.  In  the  case  of  finite  a,  the 
ranks  of  G  and  H  are  bounded  by  a  —  ^  where  i  is  the  length  of  y. 

Thus,  G  and  H  define  distinct  multisets  on  the  two  structures: 

|G®(a,  c)  :  c  G  c)  =  1}  92^  d)  :  d  e  d)  =  l}. 

As- a  consequence  there  exists  r  G  F  such  that 

#{c  G  G®(a,  c)  =  r  A  F®  (d,  c)  =  1} 

9^  #{d  G  :  G®'(6,  J)  =  r  A  d)  =  1}. 

This  implies  that  there  exist  natural  numbers  mi , . . , ,  such  that 
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3>m,  y)  =  rA  y)  =  1]  but 

not  =  =  1] 

(or  vice  versa).  Player  I  wins  by  the  following  strategy:  in  his  first  i  moves 
he  selects  appropriate  sets  yli , . . . ,  C  A  oi  cardinalities  mi , . . . ,  mi  so  that 
G'^{a,  c)  —  r  and  c)  =  1  for  the  tuples  c  =  ci, . . . ,  q  with  Ci  e  Ai.  By 

induction  on  i  it  follows  easily  that  whatever  sets  Bi, . .  .Bi  C  B  are  chosen 
by  Player  II  in  these  first  i  moves,  Player  I  can  pebble  elements  di, . , . ,  such 
that  G®'(6,d)  r  or  H'^\b,d)  ^  1.  Since  both  G  and  H  have  ranks  <  a,  the 
induction  hypothesis  implies  that  Player  I  wins  the  remaining  game,  and,  in  the 
case  of  finite  a,  that  he  wins  the  remaining  game  in  a  —  £  moves.  □ 

Lemma  5.9  If  Player  II  wins  the  C^-game  on  (fin(3!)),  d)  and  (fin(3)'),  6),  then 
she  also  wins  the  T^-game  on  (S),d)  and  (5)^,6). 

Proof  For  fixed  structures  the  positions  in  both  games  are  given  by  the 

tuples  d,  b  of  pebbled  elements.  Since  the  winning  conditions  of  the  two  games  are 
identical  it  suffices  to  show  the  following:  Suppose  that  Player  II  has  a  winning 
strategy  for  the  C*-game  from  position  (d,  6).  Then  Player  II  has  a  strategy  for 
one  move  of  the  T^-game  from  position  (d,  b)  to  reach  a  position  from  which  she 
again  has  a  winning  strategy  for  the  C^-game.  It  then  follows  that  also  in  the 
T*-game,  Player  II  can  forever  maintain  the  condition  that  the  pebbled  elements 
define  a  partial  isomorphism  between  the  primary  parts. 

Suppose  that  Player  I,  in  the  r*^-game  from  position  (d,  6),  starts  by  selecting 
pebbles  jiy...,ji  and  defining  a  function  f  :  A^  R,  By  the  assumption. 
Player  II  wins  the  C^-game  from  (d,6).  Thus  (fin(lX)),d)  (fin(2)'),  6).  By 

Theorem  5.4,  this  implies  that,  for  for  every  -equivalence  class  G  and  every 
j  <  Ar,  we  have  that 

#{c  6  A  :  (fin(®).Sf)  €  C}  =  #{c(  e  B  :  e  C}. 

Repeating  the  argument,  we  get  that  for  every  equivalence  class  G  and  every 
J  —  jii  ’  •  •  ^  jt 

#{c  6  :  (fin{®),  af)  €  C}  =  #{d  6  B‘  :  (fin(®')>*f)  £  C}. 

Thus,  there  exists  a  bijection  jr ;  A*  -+  B*  such  that  for  all  c  e 

(fin(S),af)  ). 

Now,  Player  II  defines  g  :  B^  ^  R  as  g  :=  f  o  tt,  and,  if  Player  I  pebbles 
d^B^y  she  answers  with  the  unique  tuple  c  £  A^  such  that  ttc  =  d.  The  resulting 
positions  are  in  the  same  G^^^ -equivalence  class,  so  Player  II  has  a  again  reached 
a  winning  position.  ^ 

Thus,  we  have  established  the  following  result. 
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Theorem  5.10  Lei  V  =  and  be  structures  in 

Mr  (91)  and  a  and  b  be  l-iuples  of  elements  o/2l  and  respectively.  The  fol¬ 
lowing  are  equivalent 

(i)  Player  II  wins  ike  T^-game  on  (lD,a)  and  fX)\b). 

(ii)  (2),  a)  and  (2)',  6)  are  equivalent 

(Hi)  Player  II  wins  the  C^-game  on  (fin(2)),a)  and  (fin(2)^),  6). 

(iv)  (fin(2)),a)  and  (fin(2)'),6)  are  C^^^-equivalent. 


5.3  Invariants 

The  descriptions  of  or  -equivalence  in  terms  of  the  A:-pebble  games 
give  rise  to  invariants  that  represent  in  a  compact  way,  by  means  of  an  ordered 
finite  structure,  the  complete  or  theory  of  a  given  finite  structure. 

The  first  such  invariants  were  found  by  Abiteboul  and  Vianu  [2].  They  were 
formulated  in  terms  of  computability  by  relational  machines  rather  than 
definability,  but  the  notions  are  very  closely  related.  With  these  invariants,  Abite¬ 
boul  and  Vianu  could  prove  that  the  logics  FP  and  PFP  coincide  (with  respect 
to  expressive  power)  if  and  only  if  Ptime  =  PsPACE.  We  refer  to  [14]  for  a  very 
nice  exposition  in  terms  of  T^^-equivalence. 

Invariants  for  (7^^^ -equivalence  have  been  defined  by  Otto  (see  [17,  40,  41]) 
and  been  used  to  prove  a  number  of  results  on  the  structure  of  fixed  point  logic 
with  counting,  on  the  relationship  of  (FP  +  C)  with  other  logics  and  on  the 
canonization  problem  with  respect  to  C^^^-equivalence. 

We  give  an  informal  description  of  -invariants.  For  ib-tuples  d,  a'  from  a 
fixed  structure  51,  we  write  d  ^  d'  to  denote  that  (51,  d)  and  (5t,  d')  are 
equivalent.  We  write  [d]  for  the  <N.-equi valence  class  of  d,  also  called  the  C^^^-type 
of  d. 

The  desired  C^^^-invariant  of  a  structure  5t  has  the  form 

l\QL)  =  {^,vu...^Vk) 

where  ©  =  (A^/^  is  an  ordered  structure  over  the  set  of  -equivalence 

classes  in  A*,  and  where  weight  functions  Vj  :  {A^ / rS)  — ^  associate  with  every 
type  [d]  the  number 

t;j'([d])  :=  #{6  G  A  :  d  -  d^}. 

With  the  game  characterization  of  (7^^^ -equivalence  it  can  be  shown  that 
both  and  a  total  order  -<  on  A*/  ^  (which  is  a  pre-order  on  A*")  can  be 
inductively  defined.  One  starts  with  an  arbitrary  ordering  -*<o  of  the  atomic 
types  in  k  variables.  At  every  stage  a  pre-order  -<i  on  A*  is  defined  such  that 
the  associated  equivalence  relation  (i.e.  d  a!  iff  neither  d  ■<{  o!  nor  W  -<i  d) 
describes  that  Player  II  can  maintain  her  winning  condition  for  at  least  i  moves. 
The  refinement  step  can  be  derived  from  Theorem  5.3:  d  d'  if  either  d  -<i  a! , 
or  d  a!  and  the  following  condition  holds: 


355 


For  the  sequence  C\  -<i  C2  ■<{ - Cr  of  ~i-equivalence  classes,  there 

exist  m  <  r  and  j  k  such  that  #{6  ^  A  :  a j  €  Cm}  <  il^{^  ^  ^  ' 
d'4  E  Cm}  and  for  all  pairs  {£,i)  <iex  (rnj)  we  have  that  #{6  G  A  : 
df  6  Cl}  =  #{6  €  A  :a'j  ^  Ci}. 

Note  that  this  refinement  process  is  a  variant  of  the  colour  refinement  rnethod 
leading  to  the  stable  colouring  of  a  graph  (see  Example  4.23). 

It  follows  from  this  description  that  the  limits  -<  and  of  this  inductive 
process  are  definable  in  (FP  +  C).  In  fact,  a  weaker  logic  is  sufficient,  namely 
fixed  point  logic  together  with  a  simple  form  of  cardinality  comparison  which  is 
captured  by  the  so-called  Rescher  quantifier. 

Definition  5.11  The  Rescher  quantifier  is  a  Lindstrom  quantifier  which  com¬ 
bines  two  given  formulae  together,  binding  a  single  variable  in  each  of  the  two 
formulae.  From  and  (p{y,z)  the  new  formula  [Resch  xy  'ipix,z),(p{y,z)] 

is  formed.  Its  semantic  is  defined  by  the  equivalence 

1=  [Resch  xy  xl){x,z),(p{y,z)]  ^ — »  {#xWx,z)]  <  #y[v?(2/,  ^)])  • 

We  write  FP[Resch]  for  the  logic  obtained  by  adjoining  the  Rescher  quantifier 
to  FP. 

Besides  the  relation  (for  equality),  (for  the  linear  order),  and  the  already 
described  weight  functions  ui, . . . ,  Vfc,  the  structure  7*(2t)  is  endowed  with  some 
additional  relations  to  make  sure  that  it  encodes  the  entire  C^^^-theory  of  21. 

Atomic  types:  For  every  atomic  type  t{xi, , , ,  ,Xk)  of  vocabulary  Tq,  7*'(2t) 
contains  a  unary  relation  Pt  :=  {[d]  G  A*/  2t  [=  t(d)}. 

Reachability  relations:  For  j  =  1,...,^:,  7*(2t)  contains  a  binary  relation 
7;y([a][d'])  which  indicates  that  the  type  [d']  can  be  obtained  from  [d],  by 
changing  the  j-th  coordinate.  In  other  words, 

Ej  :=  {([a][a'])  :  (36  €  A)  a)  ~  a'}  =  {([a][af])  '.be  A}. 

Permutations:  For  every  permutation  cr  E  ,  we  incorporate  a  binary  relation 
Ta  :=  {([d][d'])  :<T(d)  ^d'}  where  cr(ai, ...,  a*)  :=  a<7(i),  •  •  • ,  «<7(jb)- 

Obviously  these  additional  relations  are  easily  definable  from  21  and  Fur¬ 
ther,  it  should  be  noted  that  there  is  some  redundancy  in  this  description  in  the 
sense  that  some  relations  are  definable  from  others. 

We  can  summarize  the  result  on  -invariants  as  follows. 

Theorem  5.12  [17,  40,  41]  For  every  k  and  every  finite  relational  vocabulary 
Ta  there  exists  a  function  ,  associating  with  every  structure  2t  G  Fin(Ta)  the 
C^f^-invariant  7*(2t)  =  (25,  vi, . . .  ,^^fe)  such  that  the  following  hold: 

(i)  The  mapping  21 »— ►  25  is  definable  in  FP[Resch]. 
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(ii)  For  every  j  <  k,  ike  weight  function  vj  :  (A^ /  ^)  — ^  N  is  definable  from 
%hy  a  counting  term  v(f)  =  #y[v?(^,2/]  with  (p  €  FP[Resch]. 

(Hi)  Qt  and  05  are  equivalent  if  and  only  /*(2t)  =  /*(05), 

Corollary  5.13  [17,  40]  For  every  class  K  C  Fin(ra),  the  following  are  equiva¬ 
lent 

(i)  iC  is  definable  in  (FP  +  C). 

(ii)  For  some  A:  G  N,  {7^(2t)  :  21  E  JC}  is  decidable  in  “polynomial  time. 

Since  the  distinguishing  power  of  the  infinitary  term  calculus  can  be 
reduced  to  C^^-inequivalence  of  the  corresponding  finite  structures,  we  obtain 
a  notion  of  -invariants.  It  turns  out  that  the  T^^-invariant  J*(2))  of  an 
arithmetical  structure  5D  can  be  represented  by  a  single  natural  number,  and 
that  J*  actually  is  an  FP^-definable  global  function. 

Theorem  5.14  For  every  k  and  every  vocabulary  T  of  arithmetical  structures 
there  exists  a  numerical  invariant  :  Mr[01]  N  with  the  following  properties 

(i)  is  FV* -definable. 

(ii)  For  all  5),®'  6  Mr  [01] 

We  sketch  the  proof.  From  Theorem  5.10  we  know  that  3)  and  2)'  are 
equivalent,  if  and  only  if  the  corresponding  finite  structures  fin(2))  and  fin(0*') 
^^TOa;-efiuivalent.  We  cannot  directly  use  the  invariant  7^(fin(2))),  due  to  the 
infinite  vocabulary  of  fin(2)).  However,  an  inductive  process,  similar  the  the  one 
defined  above,  can  be  used  to  work  directly  with  2),  rather  than  with  fin(2)). 
It  is  obvious  that  FP[Resch]  and  the  simple  applications  of  counting  needed  for 
defining  the  weight  functions  can  be  simulated  in  FP*  with  secondary  part  01. 
Further  an  ordering  (or  pre-ordering)  on  the  primary  part  induces  a  ranking  (or 
pre-ranking)  of  points:  just  assign  to  a  point  the  number  of  smaller  points.  We 
thus  obtain  an  FP*  definable  function,  mapping  every  2)  6  Mr  [01]  to  a  ranked 
arithmetical  structure  that  characterizes  2)  up  to  -equivalence.  Finally  we 
can  use  the  same  techniques  as  in  the  proof  of  the  Coding  Lemma  in  the  previous 
section  to  encode  this  structure  by  a  natural  number. 

With  these  invariants,  we  easily  get  a  converse  for  Proposition  4.25  for  the 
case  that  01  =  PTA. 

Theorem  5.15  A  class  IC  C  Fin(To)  is  FF* -definable  over  PTA,  if  and  only  if 
JC  is  (FP  -f-  C)-definable. 

Proof  The  only-if  direction  has  already  been  established.  Suppose  K  is  (FP  -|- 
C)-definable.  This  and  the  FP*-definability  of  J*  imply  that  the  class  { J^(2lai)  : 
21  6  /C}  C  N  is  decidable  in  polynomial- time  and  therefore  expressible  by  a  basic 
PTA-predicate.  Since  is  FP*-definable,  the  result  follows.  □ 
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6  Asymptotic  probabilities 

Among  the  most  beautiful  results  in  finite  model  theory  are  the  limit  laws  (in 
particular  0-1  laws)  for  various  logics  and  probability  distributions  (see  [11]  for 
a  survey). 

We  consider  similar  questions  for  metafinite  structures,  with  fixed  secondary 
part.  It  turns  out,  that  limit  laws  hold  only  in  rather  restricted  cases.  Neverthe¬ 
less,  it  is  interesting  to  investigate  and  classify  these  cases. 

Probability  distributions.  Fix  a  vocabulary  T  =  (Ta, where  Ta  and 
Tu,  are  finite.  Furthermore,  fix  a  Tr-structure  together  with  a  probability 
distribution  i/  on  the  universe  R.  Finally,  fix  for  every  n  6  N  a  probability 
distribution,  over  the  finite  set  of  Ta-structures  with  universe  n={0,...,n-l}. 
In  this  paper,  will  always  be  the  uniform  distribution,  giving  equal  probability 
to  all  structures. 

We  define,  for  every  n  G  N,  a  measure  A„  on  the  space  Sn  of  metafinite 
structures  2)  G  Mr [9*1]  whose  primary  part  has  universe  n.  The  measure  is 
defined  by  the  following  experiment: 

-  The  primary  part  21  of  2)  is  chosen  according  to  the  distribution  fin  • 

-  For  every  function  symbol  w  €  Tu,  and  every  tuple  d,  the  value  ii;'^(d)  is 
selected  according  to  distribution  u. 

Thus  the  measure  An  defined  in  this  way  on  Sn  is  the  product  measure  of 
the  uniform  distribution  fin  over  the  finite  set  of  primary  parts  with  the  product 
Ylwer  copies  of  i/.  We  denote  the  sequence  Ai,  A2, . . .  by  A.  For  any 

class  C  C  Mr  [9^]  of  metafinite  structures  we  let 


A„(C)  :=  XniCnSn). 

We  now  can  define  the  corresponding  probabilities  of  a  sentence  (p  in  any 
logic  L  of  metafinite  structures  as  follows: 

An(¥^)  =  An({2)G5n:2)N^)}* 

If  the  limit  A(v?)  =  linVi-^oo  An(<^)  exists,  we  call  it  the  asymptotic  probability 
of(p.  If  this  limit  exists  for  every  sentence  of  L,  then  we  say  that  the  convergence 
law  holds  for  L  with  respect  to  A.  If,  in  addition,  every  sentence  has  asymptotic 
probability  either  0  or  1,  we  say  that  the  0-1  law  holds  for  L  with  respect  to  A. 

There  are  also  other,  weaker  notions  of  limit  laws,  such  as  the  existence  of 
Cesaro  limits 

liih  (Ai(y?)  -I-  A2(v?)  H - h  \n{^))ln 

n— t-oo 

or  the  weak  convergence  law  (introduced  by  Shelah  who  called  it  the  very 
weak  0-1  law  [44]),  saying  that 

lim  An+i((c>)  -  A„(¥>)  =  0. 

n— t-oo 


358 


It  is  clear  that  already  very  little  of  arithmetic  present  in  suffices  to  refute 
the  convergence  law.  If  contains  the  natural  numbers  and  parity  is  definable, 
and  if  we  have  summation  over  multisets  then  we  can  say  that  the  number  of 
elements  of  21  is  even.  This  holds  even  for  the  trivial  situation  that  Tq  =  =  0. 

Thus,  the  question  whether  a  convergence  law  or  a  0-1  law  holds,  is  interesting 
only  for  rather  limited  secondary  parts  9i.  In  the  sequel,  we  consider  classes  of 
simple  metafinUe  structures^  with  various  cases  of  91,  T  and  i/. 


6.1  The  uncountable  case 

It  should  be  noted  that  A„(^)  is  not  defined  in  all  situations.  In  fact,  if  (p  is 
infinitary,  then  the  set  {2)  €  Sn  :  2)  |=  need  not  be  measurable.  We  show 
this  by  means  of  an  example  (that  uses  the  axiom  of  choice  and  the  continuum 
hypothesis). 

Proposition  6.1  Let  91  =  {R,  0, 1,  |,  -f,  •,  <),  where  R  is  the  real  interval  [0, 1] 
and  -{-  is  addition  modulo  1.  LetTa  =  0,Ty}  =  {c}  where  c  is  nullary.  Then,  even 
for  n  =  1,  there  is  no  probability  distribution  on  [0, 1]  under  which  every  sentence 
-^ooo;  defines  a  measurable  subset  of  Sn  and  every  singleton  has  probability  0. 

Proof  It  is  known  that,  on  the  basis  of  the  axiom  of  choice  and  the  continuum 
hypothesis,  there  exists  no  probability  distribution  on  [0, 1],  giving  probability  0 
to  singletons,  such  that  all  subsets  of  [0, 1]  are  measurable. 

It  therefore  suffices  to  show  that  for  every  set  X  C  [0, 1]  there  exists  a 
sentence  xj)x  €  such  that  for  structure  2)  £  Mr  [91] 

'D  \=ipx  if  and  only  if  c'^  £  X. 

Every  real  number  r  £  [0, 1]  can  be  approximated  by  sequences  {an)ne<^ 
and  of  dyadic  rational  numbers  (i.e.  rationals  whose  denominators  are 

powers  of  two)  such  that  <  r  <  and 

lim  On  =  lim  =  r. 

n— •■oo  n-+oo 

Every  dyadic  rational  in  [0, 1]  is  representable  by  a  basic  weight  term  in  our 
language.  Thus,  in  we  can  form  the  sentence 

:=  /\  (fln  <  C  A  C  <  bn) 

n£w 

expressing  that  that  c  =  r.  Now  the  sentence  'ipx  :=  ^ rex  Pr  asserts  that 
c  £  X,  which  is  what  we  wanted  to  prove.  □ 


Even  though  atomic  formulae  over  91  define  very  simple  sets,  measurability 
need  not  be  preserved  under  unrestricted  conjunctions  and  disjunctions  available 
i^  ^oow  Fortunately,  there  exist  reasonable  conditions  on  a  logic  L  and  a  sec¬ 
ondary  part  91  such  that  all  T-definable  model  subclasses  in  Sn  are  measurable. 
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Definition  6.2  Let  !Hbe  a  structure  over  Tr  and  1/  a  probability  distribution  on 
R.  We  say  that  91  has  measurable  atoms  with  respect  to  u  if  every  (first-order) 
atomic  formula  ^(2^1, . .  • ,  2:^)  of  vocabulary  Tr  defines  a  measurable  set  so  that 
i/{{u  6  :  91  (=  ¥’(ii)})  is  defined. 

Proposition  6.3  If  91  has  measurable  atoms  with  respect  to  i/,  and  every  L- 
formula  is  equivalent  to  a  formula  in  then,  for  every  n,  every  L-definahle 

model  class  in  Sn  is  measurable  with  respect  to  A^. 

Proof  Fix  a  primary  part  21  with  universe  n  and  let  5'(2l)  be  the  set  of  structures 
S)  6  5n  with  primary  part  2t.  Since,  for  fixed  n,  there  are  only  finitely  many 
primary  parts,  it  suffices  to  show  that  the  set  {$)  G  5(21)  :  2?  |=  is  measurable 
for  every  fixed  21  and  every  sentence  V’  G  Then  {2)  G  5n  :  2)  f=  is  a 

finite  union  of  measurable  sets  and  thus  measurable. 

It  suffices  to  prove  the  claim  for  the  expansion  of  the  structure  21  with  names 
for  all  elements  of  A.  We  therefore  suppose  without  loss  of  generality,  that  every 
element  of  21  is  an  individual  constant.  On  5(2t),  the  logic  then  admits 
the  elimination  of  quantifiers  and  of  all  primary  relation  and  function  symbols, 
except  the  constants:  Every  quantifier  3x13  is  replaced  by  ^^ery 

primary  term  by  the  name  of  its  value  and  every  primary  atomic  subformula 
Q(a)  by  its  truth  value.  Thus  the  given  sentence  V'  is  equivalent  to  a  quantifier- 
free  sentence  (p.  Since  weight  terms  ii;(a)  are  random  variables  with  respect  to 
the  distribution  1/  and  since  91  has  measurable  atoms  with  respect  to  i/,  it  follows 
that  for  every  atomic  formula  a  =  P(u^i(di), . . . ,  Wk(ak))  that  may  occur  in  (p, 
the  set  {2)  :  2)  G  5(2t)  A  2)  |=  a}  is  measurable.  Since  the  measurable  sets  are 
closed  under  complementation  and  under  countable  unions  and  intersections  the 
claim  follows.  □ 


Examples.  We  now  consider  some  specific  examples  for  91,  u,Ta  and  T«;  such 
that  the  existence  of  a  convergence  law  or  a  0-1  law  for  first-order  logic  can  be 
easily  reduced  to  known  results  in  finite  model  theory.  We  write  FO  for  first- 
order  logic  in  the  classical  sense,  and  FO*  for  its  extension  to  first-order  logic  of 
metafinite  structures. 

One  unary  weight  function  into  an  uncountable  linear  order.  Let  91  = 
([0, 1],  <)  with  the  uniform  measure  on  [0,1],  let  Ta  be  an  arbitrary  finite  rela¬ 
tional  vocabulary  and  =  {u^}  with  w  unary. 

For  any  metafinite  structure  2)  =  (2t,  91,  {u;})  C  Mr  [91],  the  weight  function 
w  defines  a  partial  order  on  A  by 

a  <b  iff  2)  1=  ii;(a)  <  w(b). 

If  2)  is  chosen  randomly,  then  almost  surely  2)  |=  \fx\fy  w{x)  ^  w{y),  so  <  is 
in  fact  a  random  total  order  on  21.  Replacing  u;(a;)  by  x  we  can  translate  every 
sentence  ^  G  FO*  to  a  sentence  ^  G  FO  such  that,  almost  surely,  2)  [=  V’  if  and 
only  if  (2t,  <)  |=  (p. 
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The  problem  is  thus  reduced  to  a  problem  on  a  class  of  random  finite  ordered 
structures. 

For  specific  results,  we  distinguish  several  cases  according  to  the  vocabulary 
Ta  of  the  primary  part: 

Tfl  =  0:  In  this  case  the  structures  have  the  form  D  =  {ti;})  and  the 

reduction  gives  a  pure  linear  order  (n,  <).  It  is  well-known  that.no  first- 
order  sentence  ip  can  distinguish  between  linear  orders  (n,  <)  and  (m,  <) 
if  both  n  and  m  are  larger  than  a  constant  no  that  depends  only  on  the 
quantifier-rank  of  (p.  Thus,  we  have  a  0-1  law  for  FO. 

However,  in  logics  with  recursion,  such  as  transitive  closure  logic  or  fixed 
point  logic,  the  presence  of  a  linear  order  suffices  to  express  that  the  structure 
has  an  even  number  of  elements,  and  we  therefore  do  not  have  any  conver¬ 
gence  law  for  these  stronger  logics.  The  same  applies  to  monadic  second-order 
logic  MSO. 

Ta  is  monadic:  Clearly,  we  no  longer  have  a  0-1  law.  The  sentence 
Va?([Vy  w{x)  <  iy(2/)]  Px) 

expresses,  that  the  elements  with  minimal  weights  satisfy  P.  This  is  true 
with  probability  1/2  in  all  cardinalities. 

However,  we  still  have  the  convergence  law,  because  of  the  convergence  law 
for  the  first-order  logic  of  random  monadic  structures  with  a  linear  order. 
This  results  appears  in  [37]  but  is  attributed  there  to  Ehrenfeucht. 

Ta  contains  at  least  one  binary  predicate.  Here  we  have  non-convergence, 
due  to  the  result  of  Compton,  Henson  and  Shelah  [12],  that  on  the  class  of 
random  ordered  graphs  there  exist  first-order  sentences  without  an  asymp¬ 
totic  probability. 

Two  unary  functions  into  an  uncountable  linear  order.  For  structures 
V  =  (21, iH,  {vjit;})  with  two  unary  weight  functions  into  =  ([0, 1],  <),  it  is 
easy  to  see  that  we  no  longer  have  a  0-1  law.  For  instance,  the  sentence 

3x3y(Vz(v(x)  <  v(z)  A  w{y)  <  w{z))  A  t;(a;)  <  w{y)), 

expressing  that  the  minimal  v- weight  is  smaller  than  the  minimal  u;- weight,  is 
true  with  probability  1/2  in  all  cardinalities.  In  fact,  we  don’t  even  have  the 
convergence  law.  With  two  weight  functions  we  can  almost  surely  interpret  two- 
dimensional  partial  orders  (i.e.  the  intersection  of  two  linear  orders),  and  it  is  a 
result  of  Spencer  [45],  that  there  exist  first-order  sentences  without  asymptotic 
probabilities  for  A:-dimensional  partial  orders,  whenever  k  >2. 

Field  of  reals  as  secondary  part.  A  different  class  of  examples  is  obtained 
by  for  the  secondary  part  the  field  of  reals  =  (IR,H-,  -,0, 1). 

Here  we  have  a  0-1  law  for  arbitrary  relational  Ta  and  arbitrary  T^ju.  This 
might  come  as  a  surprise,  but  it  is  true  for  rather  trivial  reasons:  Take  any  pair  of 
basic  weight  terms  F(x),  0(y).  Then  almost  surely  either  V  [=  F(x)  =  G{y) 
or  S)  [=  VxVy  F{x)  ^  G{y).  Thus,  the  secondary  part  almost  surely  provides  no 
information  at  all,  so  the  0-1  law  holds  whenever  it  holds  on  finite  structures. 


6.2  The  countable  case 

The  other  interesting  case  is  when  the  secondary  part  is  countable.  We  may 
assume  that  its  universe  is  the  set  of  natural  numbers.  Then  v  is  given  by  a 
sequence  of  nonnegative  reals  such  that  Yl^=oPn  =  1  =  j/({n}).  We 

first  show  that  one  gets  a  strong  form  of  non-convergence  even  in  very  simple 
cases.  As  above,  A  =  Ai,  Ai, . . .  is  the  sequence  of  distributions  induced  by  u. 

Definition  6.4  A  distribution  decreases  rapidly  if  lim„_^oo  =  0- 

An  example  of  a  rapidly  decreasing  distribution  is  the  Poisson  distribution 
:=  with  the  mean  value  fi. 

Proposition  6.5  Suppose  that  Ta  =  Tr  =  0  and  consists  of  one  unary 
function  name  w,  and  let  X  be  induced  by  a  rapidly  decreasing  distribution  i/. 
Then  the  sentence 

(p  =  3x\/y{y  zjzx^  w{x)  rfi  w{y)), 

has  no  asymptotic  probability  with  respect  to  A.  Even  the  Cesaro  probabilities 

Xki<p)  =  Pi(v’)  +  •  •  •  + 
do  not  converge. 

Proof.  We  start  with  preliminary  observations.  Since  Pn+i/pn  =  0  tends  to  0, 
for  every  c  <  1,  there  exists  m  =  m(c)  such  that  Pn+i/Pn  <  c  for  all  n  >  m. 
Thus  we  may  assume  without  loss  of  generality  that  pn+i  <  Pn/4  for  all  n. 

The  sum  Ylj>nPj  j Pn  =  ^  converges  to  1  as  n  grows  to  infinity.  Indeed,  for 
every  e:  >  0,  there  exists  a  positive  c  <  1  such  that  (c/(l  —  c))  <  e.  Let  m  =  m{c) 
be  as  above  and  suppose  that  n  >  m.  We  have 

^  <  y;  c^-"  ==  1+ c/(i  -  c)  <  1 + e. 

j>n 

Finally,  e“^  <  (1  -p)^/P  <  if  0  <  p  <  1/2.  Indeed,  apply  the  Mean  Value 
Theorem  to  the  function  f{t)  =  —  log(l  —  t)  on  the  interval  [0,p].  There  is  a 
point  i  €  (0,p)  such 

/(p)  “  /(O)  =  -  log(l  -  p)  =  (p  -  0)f\t)  =  p/(l  - 1). 

Since  p  <  p/(l—<)  <p/(l-p)  <  p/(l-l/2)  =  2p,  we  have  p<  —  log(l-p)  < 
2p  and  therefore  e~^^  <  1  —  p  <  e~^ .  Now  raise  the  terms  to  power  1/p. 

Now  we  are  ready  to  prove  the  proposition.  The  idea  is  as  follows.  Let  p  =  p,- 
and  M  =  [1/pJ,  so  that  Mp  — ►  1  and  M  grows  much  faster  than  i.  We  will 
check  that  the  probabilities  AAf(3a;![u;(a;)  =  i])  converge  to  a  positive  num¬ 
ber  and  therefore  the  probabilities  Am (¥>)  have  a  positive  limes  inferior.  Fur¬ 
ther,  let  N  =  [l/>/p*+iP»J ,  so  that  Npi  — >  oo  and  iVp,+i  0.  We  will  check 
that  the  probabilities  Ajv(3a:[i£;(f)  >  i])  converge  to  zero  and  the  probabilities 
\N(\/j^-3x\[w{x)  =  j])  converge  to  zero.  Therefore  probabilities  Ajv(^)  con¬ 
verge  to  zero,  because,  for  every  n. 


362 


•^n(¥’)  <  ^n{\J  3®![tu(x)  =  j])  +  A„(3®[u;(x)  >  i]). 

)<i 

Now  let  us  do  the  necessary  computations. 

Part  1.  Let  n  range  over  the  interval  [M,  2M]  and  c(p)  =  (1  —  —  e”^,  so 

that  e(p)  =  o(l)  as  p  tends  to  0.  We  have 

An(¥>)  >  A(3!xKx)  =  i])  >  np(l  >  np[(l  -p)i/'’]"P 

^  =  np[e-i  +  e(p)]"P  >  Mp[e-^  +  +  o(l). 

It  follows  that 

liminf Xt(¥>)  >  liminf X2m(v’)  >  +  M{e-^  +  o(l))]  > 

Part  2.  Let  n  range  over  [iV  +  1, 18A^].  We  have 
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Notice  that  Npj  —  Npj^i  >  1  if  j  <  i.  Indeed,  Npj  >  Npi  >  y/pi/pi^i  >  2 
because  every  Pm+i  <  Pm/^>  Further,  Npj^i  <  Npj/A.  Hence  Npj  —  Npj^i  > 
(3/4)iVpj  >  1.  Therefore 

oo 

A„(y3®!Hs)  =  i])<18-  ^  me-”, 

j  m=LJVpiJ 

which  converges  to  0  when  i  grows  to  infinity  because  the  series  Tne~^ 

is  convergent  and  Npi  <  Pi/y/pipi^  =  y/pijpi^  — ^  oo-  Consequently,  A„(v?)  = 
o(l)  and  therefore 

1  N 

liminf(x)fe(¥>))  <  X18n(¥>)  <  °(1))  ^ 

m=:l 

□ 

However,  there  is  a  weaker  form  of  limit  law,  introduced  by  Shelah,  which  .is 
of  interest  for  this  case. 

Definition  6.6  We  say  that  a  class  of  sentences  L  satisfies  the  weak  convergence 
law  with  respect  to  A  =  (An)neN  if  for  a\\  'ip  E  L  we  have  that 

lim  Xn+i{rp)  -  An(^)  =  0. 
n— t-oo 

For  instance,  it  has  been  proved  by  Shelah  [44],  that  first-order  logic  satisfies 
the  weak  convergence  law  on  ordered  random  graphs  and  also  on  a  random 
binary  function.  We  can  prove  a  similar  results  for  monadic  classes  of  metafinite 
structures  with  an  arbitrary  countable  secondary  part. 

Theorem  6.7  Let  fR  be  any  structure  with  universe  N,  endowed  with  an  arbi¬ 
trary  probability  distribution!/,  and  letTa  andT^u  be  unary.  Then  for  the  induced 
sequence  A  of  probability  distributions,  first-order  logic  satisfies  the  weak  conver¬ 
gence  law. 

Proof  Let  2)  =  {%fR,W)  and  2)^  =  (55,91,  W')  be  two  structures  in  Mr[9i]. 
Recall  that  for  a  €  ^  and  b  E  B, 

(2),  a)  -0  (5)',  6) 

means  that  the  function  p  :  a  6  is  a  partial  isomorphism  from  2)  to  2)^  i.e. 
that  a  and  b  satisfy  the  same  Ta-relations  over  21  and  respectively,  and  that 
the  weight  functions  map  a  and  b  to  the  same  values  of  N.  For  every  m  €  N,  we 
say  that  a  '^o-^quivalence  class  C  is  m-bounded,  if  <  m  for  all  w  E  Tw 

and  (2),  a)  €  C. 

The  structures  2)  and  2)'  are  A:-equivalent,  i.e.  cannot  be  distinguished  by 
formulae  of  quantifier  depth  k,  if  every  ^o-equivalence  class  C  contains  the  same 
number  of  elements  in  2)  and  2)^  or  more  than  k  elements  in  both  structures.  This 
can  be  proved  by  a  straightforward  application  of  Ehrenfeucht-Fraisse  games. 
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For  every  €>  0  take  a  large  enough  natural  number  m  so  that  pi  >  l~e. 
Given  choose  no  large  enough  such  that  for  every  n  >  no,  a  random  D  € 
contains,  with  probability  at  least  1  —  more  than  k  elements  in  every  m- 
bounded  '^o-equivalence  class. 

The  process  of  drawing  a  random  structure  ©  €  5n+i  can  be  described  as 
follows:  first  we  choose  a  random  structure  S'  €  ;  then  we  add  a  new  element 

a  and  determine  at  random  the  truth  values  of  atoms  Pa  for  P  £Ta  and  the 
values  of  the  weight  terms  w[a)  for  w;  G  With  probability  at  least  (1  -  ef 
(where  I  =  |Tu;[),  the  '^o "Equivalence  class  of  u  is  m-bounded.  As  a  consequence, 
if  n  >  no,  then  iD  and  3)'  differ  by  an  element  that  almost  surely  belongs  to 
a  class  with  more  than  k  representants  in  both  structures.  Thus,  2)  is  almost 
surely  ^-equivalent  to  2)'. 

Since  k  was  arbitrary,  it  follows  that  that  for  every  first-order  formula  ^ 
lim  A„(^)  -  Xn-^-iW  =  0. 

n-^oo  f  V  / 

□ 

Remark.  With  the  same  argument,  the  weak  convergence  law  also  holds  for 

ru> 
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Automatic  Presentations  of  Structures 


Bakhadyr  Khoussainov  and  Anil  Nerode 


Introduction 

In  this  paper  we  introduce  the  systematic  study  of  presentations  of  algebraic 
structures  by  finite  automata.  We  call  these  automatic  structures. 

The  study  of  recursive  presentations  of  algebraic  structures  was  initiated 
by  Proehlich  and  Shepherdson  [8],  Rabin  [14],  and  Mal’cev  [11].  Since  then, 
recursive  algebra  has  been  an  active  area  of  study  by  researchers  associated 
with  Nerode  in  the  U.  S.  and  Ershov  in  Russia.  A  recursive  structure  is  a 
countable  structure  equipped  with  Turing  machines  for  deciding  equality  and 
the  other  atomic  relations.  The  ordering  of  rationals,  vector  spaces  over  rational 
numbers,  absolutely  free  algebras,  free  groups,  and  finitely  presented  algebras 
with  decidable  word  problems  have  obvious  recursive  presentations.  Recursive 
Boolean  algebras,  linear  orderings,  abelian  groups,  vector  spaces,  fields,  lattices, 
and  other  structures  have  been  studied  extensively. 

In  the  late  1980’s  Nerode,  Remmel,  and  Cenzer  [15]  [16]  developed  a  cor¬ 
responding  theory  of  p-time  structures.  These  are  recursive  structures  which 
are  presented  by  polynomial  time  recursive  functions.  This  has  also  become  an 
active  area  of  research.  For  example,  Remmel  proved  that  any  recursive,  purely 
relational,  structure  is  recursively  isomorphic  to  a  p-time  structure,  and,  as  a 
corollary,  that  any  recursive  Boolean  algebra  has  a  polynomial  time  presentation. 

In  this  paper  we  further  restrict  the  recursive  functions  of  the  presentaions 
and  insist  they  be  given  by  finite  automata.  We  obtain  a  very  fine  grain  theory  of 
automatic,  or  automaton  presentable  structures.  That  is,  these  are  structures 
provided  with  finite  automata  for  deciding  equality  on  the  domain  and  the  atomic 
relations  of  the  structure. 

We  note  that  automatic  groups  in  a  closely  related  sense  are  an  active  object 
of  study  ([6])  growing  from  the  need  to  have  feasible  calculations  in  3- manifold 
theory.  Epstein,  Cannon,  Thurston  have  developed  the  theory  of  automatic 
groups  motivated  by  performing  computations  on  groups  associated  with  3- 
manifolds.  They  consider  finitely  generated  groups  with  generators  pi, . . . 
Each  generator  gi  naturally  defines  a  unary  operation  fi  on  the  domain  of  G 
by  the  right  multiplication,  that  is  fi{x)  —  xgi^  where  x  ^  G.  Thus,  with  the 
finitely  generated  group  G  one  can  associate  the  unary  structure  ((?, /i, .  -  • ,  /n)* 
They  call  the  group  G  automatic  ([6])  if  the  corresponding  unary  structure 
(G,/i, . . . ,/n)  is  automatic  in  our  sense  below.  They  do  not  impose  the  re¬ 
quirement  that  the  binary  group  operation  be  described  by  a  finite  automaton. 
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They  prove,  for  instance,  that  any  automatic  group  is  finitely  presented  [6]. 
Automatic  groups  in  the  sense  of  Epstein,  Cannon,  and  Thurston,  regarded  as 
unary  structures  as  above,  are  automatic  in  our  sense. 

Research  on  automatic  structures,  unlike  research  on  recursive  and  p-time 
structures,  concentrates  on  positive  results.  The  results  of  Epstein, Cannon, 
Thurston  show  the  usefulness  of  automaton  presentations.  If  one  has  an  automa¬ 
ton  presentation  of  a  structure  ^4,  one  can  perform  automaton  computations  on 
the  structure.  As  an  example,  let  Ps  be  the  following  problem:  is  there  a  recur¬ 
sive  procedure  which,  when  applied  to  a  first  order  definition  of  a  relation  S  on 
A,  yields  an  algorithm  for  deciding  P.  To  solve  this  problem,  we  want  to  find 
presentations  of  A  in  which  computations  of  atomic  relations  are  governed  by 
finite  automata.  In  such  a  presentation,  these  computations  can  be  performed  in 
real  time.  If  we  find  such  a  presentation,  we  can  transform  the  problem  Pg  into  a 
problem  about  finite  automata.  Since  finite  automata  possess  many  decidability 
properties,  we  can  deduce  the  decidability  of  many  problems  such  as  Ps-  This, 
in  turn,  leads  to  consideration  of  the  complexity  of  problems  about  automatic 
structures. 

Here  are  several  basic  questions. 

-  What  is  an  automatic  presentation? 

-  Which  structures  are  automaton  presentable? 

-  What  does  automaton  presentablity  say  about  the  structure? 

-  What  are  the  easiest  standard  structures  that  have  or  lack  automaton 
presentations? 

-  What  is  the  complexity  of  problems  formulated  over  automatic  structures? 

We  define  the  notions  of  strongly  automatic,  automatic,  and  asynchronously 

automatic  presentations.  We  provide  many  examples  of  automatic  presentations 
of  linear  orderings,  vector  spaces,  abelian  groups,  permutation  structures,  etc. 
Though  these  examples  are  not  startling,  they  show  that  automatic  presenta¬ 
tions  play  a  basic  role  underlying  many  recursive  and  polynomial  time  structures. 
We  obtain  an  algebraic  characterization  of  automatic  structures,  by  introduc¬ 
ing  many  sorted  finite  automata.  This  leads  to  a  natural  generalization  of  the 
Myhill-Nerode  theorem  charcterizing  finite  automaton  recognizable  languages 
by  congruences  of  finite  index.  We  also  characterize  structures  which  possess 
strongly  automatic  presentations.  At  the  end  we  discuss  automatic  isomorphism 
types. 


1  Preliminary  Definitions 

Consider  structures  of  the  form  (A,  , . . . ,  ,P^° , . . . ,  PJ"* ,co, . . . ,  ct), 

where  A  is  the  domain,  each  is  an  operation  of  arity  ni  on  A,  each  Pj”^  is 
a  predicate  of  arity  rrij  on  A,  and  each  constant  ci  belongs  to  A.  We  suppose 
that  the  domain  A  is  at  most  a  countable  set.  The  sequence 


is  called  the  signature  of  the  structure  A.  Given  a  structure  A  of  signature, 
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form  a  new  structure 


Ar  =  {A, f;«+\  . . . , F"‘+\ Fo™", . . . , pr*, CO, . .  .,ct), 

where  for  all  ai,...,o„,.+i  6  A,  . . . ,a„j+i)  iff  = 

a„.+i.  This  makes  the  structure  Ar  relational.  We  often  identify  the  structure 
A  with  Ar. 


Let  S  be  a  finite  alphabet.  Form  the  set  S*  of  all  finite  words  of  the  al¬ 
phabet  E.  A  (nondeterministic)  finite  automaton  over  the  alphabet  E  is 
a  quadruple  Q  =  (5,/,  AjF),  where  S  is  the  finite  non-empty  set  of  states, 
/  is  a  non-empty  subset  of  S,  called  the  set  of  initial  states,  A  C  5  x  E  x  5 
is  a  non-empty  set,  called  the  transition  table,  F  is  a  subset  of  S',  called 
the  set  of  final  states.  Thus,  A  can  be  viewed  as  a  mapping  from  5  x  E  to 
P{S)  =  {S"  1  5'  C  S'}.  The  mapping  A  can  be  extended  naturally  to  a  mapping 
from  S'  X  E*  to  P{S).  If  there  is  no  confusion,  denote  this  extension  by  A  too. 
For  convenience  we  present  a  finite  state  automaton  fl  as  a  directed  graph.  The 
nodes  of  the  graph  are  the  states  of  the  automaton  fl.  The  edge  relation  E  on 
the  nodes  is  defined  as  follows.  There  exists  an  edge  connecting  node  s  with 
node  5'  if  and  only  if  (s,a,  5')  €  A  for  some  cr  G  E.  We  label  this  edge  cr.  Thus, 
E  =  {(s,  5')  I  3flr  G  E((5,  (t,  5')  G  A)}.  The  automaton  Cl  accepts  cri . . .  if  and 
only  if  there  exists  a  path  SQ...Sn  in  the  graph  presenting  the  automaton  such 
that  So  €  So,  Sn  G  F,  and  for  each  i  {si,ai,Si+i)  G  A.  We  call  such  a  path  a 
computation  of  0  on  input  cti  ...an.  The  behavior  of  the  automaton  Cl  is  de¬ 
fined  as  the  set  L{Cl)  of  all  words  accepted  by  Cl.  A  set,  or  equivalently  language, 
F  C  E*  is  finite  automaton  (FA)  recognizable,  or  simply  recognizable,  if 
there  exists  a  finite  automaton  Cl  such  that  D  =  L{Cl).  It  is  well  known  that  the 
set  of  all  finite  automata  recognizable  sets  forms  a  Boolean  algebra  and  that  the 
emptiness  problem  for  finite  automata  is  decidable. 

Dealing  with  an  automatic  structure  A,  it  is  necessary  to  have  an  automaton 
that  recognizes  the  domain  of  the  structure.  To  do  this  we  have  to  present 
elements  of  the  structure  as  words  of  a  finite  alphabet.  Thus,  suppose  that  a 
structure  A  is  given.  An  automatic  presentation  of  the  domain  of  A  is  a 
surjective  mapping  u  :  D  A,  where  D  is  a  recognizable  subset  of  E*.  If 
a  G  A  and  p(a)  =  a,  we  say  that  a  presents  the  element  a.  Having  the  mapping 
p,  we  formulate  three  problems. 

1.  Find  a  procedure  which,  for  any  two  words  a,  P  e  D  decides  whether 

i/(a)  =  u{l3). 

2.  Find  a  procedure  which,  for  each  predicate  Pp^ ,  and  all  cti , ... ,  G  D, 

decides  whether  Pp^  •  •  • )  holds. 

3.  .. Find  a  procedure  which  for  each  operation  and  all 

ai, . . .  jQni+i  ^  D  decides  whether  {p{ai), . . . ,  !/(«„. 4.1))  holds. 

Informally,  a  presentation  p  is  an  automatic  presentation  of  A  if  there  ex¬ 
ist  automata  for  deciding  the  above  three  problems.  This  considerations  suggest 
the  definition  of  automata  recognizable  relations  over  E*. 
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1.  n- variable  Strong  Automata. 

The  intuitive  meaning  of  the  concept  of  n- variable  strong  automaton  is  as 
follows.  The  inputs  for  such  an  automaton  are  tuples  of  words  over  H.  Given 
an  input,  the  automaton  acts  on  each  component  of  the  input  exactly  as  a  finite 
automaton.  Computations  at  different  components  of  the  input  are  independent. 

Definition  1.1  A  strong  n- variable  automaton  Qn  on  E  is  a  system 
(5,  jSo?  where  S,  So,  ^  are  as  for  finite  automata  and  F  is  a  subset  of  S^, 

called  the  set  of  hnsd  states. 

Let  (ai,...,a„)  €  (S*)”  and  let  cKj  be  i  =  1, ...,n.  A  se¬ 

quence  (5ii, . . . ,  Sni),  (512, . . . ,  5n2),  •  •  • ,  (^imi ,  •  •  • , 5nm„)  is  a  Computation  of 
the  automaton  fin  on  (ai, . . . ,  c^n)  if  (^n, .  • . ,  5ni)  6  Sq^  and  for  each  the 
sequence  Sii , . . . ,  Sirm  is  a  computation  of  fin  on  the  component  ai  according 
to  the  transition  table.  The  n-tuple  (ai, . . . ,  an)  is  accepted  by  the  n- variable 
strong  automataion  Dn  if  there  exists  a  computation 

(sil, . .  . ,  5nl),  (512,  •  •  • ,  5n2),  •  •  •  ,  (^Imi ,  •  •  • ,  ^nrrin) 

of  the  automaton  on  the  n-tuple  such  that  (simi ,  •  •  • ,  5nm„ )  €  F.  A  relation  L 
of  airity  n  on  S*  is  strongly  recognizable  if  there  exists  an  n-variable  strong 
automaton  Hn  on  E  such  that  the  set  of  all  7i-tuples  accepted  by  this  automaton 
is  exactly  L. 

2.  71- variable  Automata.  Let  E  be  a  finite  alphabet.  Suppose  that  the  symbol 
O  does  not  belong  to  E.  Take  words  ai  =  an  . . . ,  aim  of  the  alphabet  E,  where 
7  =  0, . . . ,  71  —  1.  The  convolution  ao  ★ . . .  ★  an-i  of  these  words  is  defined  in 
the  following  way.  If  for  alH,  j  <  7i  ni  =  Tij,  then  the  convolution  is 

(ffoi,-..  ,  11),  •  •  • ,  (^^Ono ,  •  •  • ,  ^n— Ino)’ 

Otherwise,  let  m  be  the  maximal  length  of  the  words  ao,...,an-i.  Add  to 
the  right  end  of  each  ai  the  necessary  number  of  symbols  O  to  get  words  of 
the  length  m.  Call  these  new  words  a^,  i  =  0, . . .  ,7i  —  1.  The  convolution  of 
these  71- tuples  is  Qq  ★  . . .  ★  convolution  is  a  word  of  the  alphabet 

(E  U  {O})”.  Thus,  for  any  n-ary  relation  R  on  E*  we  can  consider  the  subset 
F*  C  (E  U  {O})”  obtained  from  R  using  convolution,  that  is, 

R  {oio  "A" . . .  ★  0Lji—\  [  (ao , . . . ,  aji—i)  ^  F)’. 

Definition  1.2  1)  An  n- variable  automaton  on  E  is  a  finite  automaton  over 
the  alphabet  (E  U  {O})”.  2)  An  n-ary  relation  R  in  E*  is  7i-recognizable, 

if  R*  is  recognizable  by  an  n~variable  automaton. 

3.  Asynchronous  Automata.  Another  recognizability  notion  for  relations  on 
E*  based  on  the  notion  of  an  asynchronous  automaton.  Let  P{Qy  be  the  set  of 
all  non-empty  subsets  of  Q,  let  P{iy  be  short  for  P({1, . .  •  ,0)^  ^  ^  ^5 

E^  =  E  U  {O}. 
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Definition  1.3  An  n- variable  asynchronous  atomation.  Q  on  E  is  a 
quadruplet  (5, 5o,  A,F),  where  S  is  the  set  of  states,  Sq  is  the  set  o/ initial 
states,  F  C  S  is  the  set  of  final  states,  and  A:Sx  (S')”  P(5)  xP(P(n)')) 

is  a  partial  mapping  called  the  transition  table  such  that: 

1.  For  all  a  e  (S')”,  s  e  S  if  a  =  (o^i, . . .  ,(Tn),  ai  —  O  for  some  i, 
A(s,cr)  =  (P,P),  and  J  e  R,  then  i  ^  J. 

2.  For  all  s  e  S,  A(5,  a)  is  undefined  if  and  only  if  (t  =  (O,  O, . . . ,  O). 

Let  us  take  words  ai  =  an,, . ,  ^  aim »  *  =  1, . . . ,  n.  The  intended  behaviour 
of  an  asynchronous  automaton  Q  on  the  n-tuple  a  =  (ai, . . .  ,an)  is  as  follows. 
The  automaton  begins  its  computation  from  an  initial  state.  Suppose  that  that 
automaton  is  in  a  state  s  e  S  and  that  the  input  is  a'  =  (ai,..  .,an)  G  (S')”. 
Consider  the  pair  {L,R)  defined  by  the  transition  table  A{s,a*)  =  {L,R),  Then 
the  automaton  non-deterministically  chooses  a  state  s'  6  L,  a  non-empty  set 
{h j  •  •  •  j u}  €  P,  and  makes  moves  on  components  , . . . ,  .  We  call  the  pair 

(s,s')  an  elementary  move  defined  by  a\  Then  one  can  naturally  define  the 
notion  of  a  computation  of  the  automaton  Q  on  input  a.  Say  that  fl  accepts  a 
if  there  exists  a  computation  of  on  a  which  begins  at  an  initial  state  and  ends  at 
a  final  state.  Thus  a  n-ary  relation  R  on  S*  is  asynchronously  recognizable 
if  there  exists  an  asynchronous  automaton  such  that  the  set  of  all  n-tuples  of 
words  accepted  by  this  automaton  is  exactly  R. 

Presentations  of  Structures.  Let  a  structure  A  of  the  signature  be  given. 
The  following  are  definitions  of  automatic  presentations.  The  next  section  gives 
many  examples. 

Definition  1.4  Let  i/  :  D  A  be  a  surjective  mapping,  where  D  is  an  automa¬ 
ton  recognizable  subset  of  S*.  The  mapping  v  is  respectively  an  automatic, 
(strong  automatic,  asynchronous  automatic)  presentation  of  A  if  u  sat¬ 
isfies  the  following  conditions: 

1.  There  exists  a  2-variable  automaton  (2-variable  strong  automaton,  asyn¬ 
chronous  automaton)  which  for  any  two  words  a,  f3  £  D  decides  whether 
v{a)  =  v{p). 

2.  For  each  j  6  {0, there  exists  an  mj-variable  automaton  (mj- 

’  variable  strong  automaton,  asynchrorious  automaton)  which  for  all  ai,...,  Om,- 
G  D,  decides  whether  ,  I'ioimj))  holds  in  the  structure  A. 

3.  For  each  i  G  {0,  there  exists  an  {ni-\-l) -variable  automaton 

({ui  +  l) -variable  strong  automaton,  asynchronous  automaton)  which,  for 
all  ai, . . .  jOno^ni+i  ^  Df  decides  whether  F^* . . .  ,i/(ani+i))  holds 
in  A, 

Ifv  is  an  automatic  (strongly  automatic,  asynchronous  automatic)  presentation 
of  the  structure  A,  then  the  pair  {A,  u)  is  a  (strongly,  asynchronous)  auto¬ 
matic  structure  and  the  structure  is  A 
(strongly,  asynchronous)  automata  presentable. 
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2  Some  Examples 

Structures  with  Unary  Predicates.  These  are  structures  of  the  form 
A=  {AyPo,..  ,,Pjn)i  where  each  Pi  is  a  unary  predicate. 

Proposition  2.1  Every  structure  with  unary  predicates  only  has  an  automatic 
presentation. 

Proof.  Let  A  =  (A, Pq? •  •  •  jPm)  be  a  structure  with  each  Pi  a  subset  of 
domain  A.  Suppose  that  for  diiferent  i^j  <  m,  Pi  n  Pj  =  0.  Consider  the 
alphabet  E  =  {0, 1}.  On  the  set  u  =  {0, 1}*  we  can  choose  pairwise  disjoint 
recognizable  sets  ^o, . . . ,  S'm  such  that  for  each  z  <  m,  card{Pi)  =  card{Si),  and 

card({0, 1}*  \  (50  U  . . .  U  Sm))  =  card{A  \  (Pq  U  . . .  U  Pm)). 

Then  any  1-1  function  from  u  :  {0, 1}*  A  such  that  z/(5t)  =  Pj  for  each  i,  is 
an  automatic  presentation  of  A. 

Suppose  that  Po,...,Pn  are  arbitrary  unary  predicates.  Then  there  exist 
pairwise  disjoint  subsets  of  A  with  the  following  property:  For  any 

i  <  m  there  exists  a  Boolean  combination  $i(Po} •  •  •  jPjfe)  of  sets  Bo,...,Bk 
such  that  Pi  =  {x  \  X  e  $i(Po) •  •  •  jPfe)}-  By  the  previous  case,  the  structure 
(A;  Bo,  •  •  • ,  P/k)  has  a  automatic  presentation  {u;So,. . .  ,Sk).  Since  recognizable 
sets  are  closed  under  Boolean  operations  the  structure 

(cj,  $o('S'oj  .  .  .  ,  5/:),  .  .  .  ,  •  •  •  j  'S'fc)) 

is  automatic  and  isomorphic  to  A.  □ 

Linear  Orderings.  Here  are  several  examples  of  automaton  presentable 
linear  orderings. 

Proposition  2.2  The  rational  numbers  with  the  natural  linear  ordering  have 
an  automatic  presentation. 

Proof.  Let  E  =  {0, 1}  and  let  P  be  a  set  such  that  alOl  6  B  if  and  only  if 
a  G  E*  and  a  does  not  have  the  subword  101.  It  is  clear  that  B  is  a  recognizable 
subset  of  E*.  Consider  the  lexicorgraphic  linear  ordering  -<1  on  the  set  E*.  This 
ordering  is  recognizable  by  a  2-variable  automaton.  Thus  the  linear  ordered  set 
(^j  1^0  is  automatic.  Let  alOl  G  D.  Then  alOl  ;:</  allOl  and  aOOlOl  ■<1  alOl. 
Hence  (B,  :<i)  does  not  have  maximal  and  minimal  elements.  Similarly  it  can 
be  proved  that  is  a  dense  linear  ordering  of  the  set  B.  It  follows  that  (B, 
is  isomorphic  to  the  the  rational  numbers  with  the  natural  linear  ordering.  □ 

Proposition  2.3  1)  For  any  natural  number  n  ^  u),  the  ordinal  has  an 

automatic  presentation.  2)  The  ordinal  uj^  has  an  asynchronous  automatic 
presentation. 

Proof.  1.  Consider  the  alphabet  E  =  {0, 1}.  To  prove  the  first  part,  define 
the  following  set  B„  =  {O^^lO^^l . . . 10*’^  |  z'l, . . . , >  1}.  The  set  Bn  is 
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recognizable.  Let  a,  /?  €  H*.  Then  a  <n  /3  if  and  only  if  there  exist  7, 71 , 72  €  S* 
such  that  a  =  7I71  and  (3  =  7O72.  One  can  construct  a  2-variable  automaton 
which  recognizes  the  relation  <„.  The  linearly  ordered  set  (D,  <n)  is  isomorphic 
to  a;”. 

To  prove  the  second  part,  define  the  following  set  D: 

D  =  {0*^  10*=^  1 . . .  0**-^  h’l , . . . ,  U  >  1,  fc  >  1}. 

Note  that  D  contains  the  set  Dn  as  a  proper  subset  and  that  D  =  Un^"* 
a,l3  e  S*.  Then  a  <  /?  if,  and  only  if,  either  a  <n  13  for  some  n  or  a  6  Dty 
13  e  Dm,  and  t  <m.  The  linearly  ordered  set  (Z),  <)  is  isomorphic  to  It  can 
be  verified  that  the  relation  <  is  recognizable  by  a  asynchronous  automaton.  □ 
The  next  proposition  shows  that  the  standard  operations  +  and  x  over 
linear  orderings  are  preserved  by  automatic  presentations.  We  leave  a  proof  of 
this  proposition  to  the  reader. 

Proposition  2.4  Let  L\  and  L2  he  linear  orderings  which  have  (asynchronous) 
automatic  presentations.  Then  the  linear  orderings  Li  4*  L2  and  Li  x  L2  also 
have  (asynchronous)  automatic  presentations. O 

Boolean  Algebras.  Let  (L,  <)  be  a  linear  ordering.  Consider  the  interval 
Boolean  algebra  Bi  generated  by  intervals  [a,  6)  =  {x  |  a  <  a;  <  6}.  Any 
element  of  this  algebra  is  a  finite  union  of  pairwise  disjoint  intervals.  Consider 
the  linear  ordering  nxu,  where  n  is  a  natural  number. 

Proposition  2.5  For  every  n  the  Boolean  Algebra  Bnxu;  has  an  asynchronous 
automatic  presentation. 

Proof.  We  prove  the  proposition  in  the  case  n  =  2.  The  remaining  case 
n  >  2  is  similar.  We  have  to  prove  that  the  Boolean  Algebra  Buj+tj  has  an 
asynchronous  automatic  presentation.  Consider  the  alphabet  {0, 1,2,0,!}.  Let 

a=[ai,6i)  U  [a2,b2)  U  ...  U  [an, bn) 

be  an  element  of  the  algebra  such  that  [ui^bi)  fl  [aj^bj)  =  0  for  i  ^  j,  and 
ai  <bi  <  .  • .  <  an  <  bn‘  There  are  several  cases. 

Case  1.  Suppose  that  bn  <  Consider  the  sequence 

i/(a)  =  eo...e6„  G  {0,1,2}*, 

where  e,-  =  1  if  i  G  \Jj-i[aj,bj)j  and  £{  =  0  otherwise. 

Case  2.  Suppose  that  ai  >  cj.  Consider  the  sequence 


z/(a)  =  02eo...€6„  G  {0,1,2}*, 

where  €i  =  1  if  a;  +  2  G  \Jj=i[aj,bj)y  and  e*  =  0  otherwise. 

Case  3.  Suppose  that  there  exists  an  m  <  n  such  that  bm  <  oj  and  am+i  > 
Thus 

a=[ai,6i)  U  [a2,b2)  U  ...  U  [am,bm)  U  ...  U  [an, bn)- 
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DsfinG  di  —  U  •••  U  s-nd  0,2  —  LJ  U  [0-71,671). 

Consider  the  sequence 

j/{a)  =  i/(ai)02i/(a2). 

Case  4-  Suppose  that  there  exists  an  m  <  n  such  that  0771  <  a;  and  6777+1  >  u. 
Thus  a  =  [ai,6i)  U  [02,62)  U  . . .  U  [0777,6777)  U  . . .  U  [077,677).  Define 

Oi  =  [oi,6i)  U  ...  U  [0777,0777  +  1), 

02  =  [u^  +  1,6777)  C  [0777+1,6777+1)  U  ...  U  [077,677). 

Consider  the  sequence  1/(0)  —  i/(oi)12z/(o2). 

We  also  put  z/(0)  =  0  and  i/{L)  =  1.  Thus  we  have  a  mapping  1/  mapping 
Boolean  algebra  into  {0, 1,2}*.  This  mapping  is  1-1.  Let  D  be  its  range. 
The  definition  of  D  implies  that  the  set  D  is  finite  automaton  recognizable.  The 
operations  fl  and  U  in  the  Boolean  Algebra  induce  the  operations  +  and  ■ 
in  the  set  D.  It  is  easy  to  prove  that  the  graphs  of  +  and  •  on  D  are  recognizable 
by  an  asynchronous  automaton.  □ 

Graphs.  Here  we  present  a  general  construction  of  automatic  graphs. 
Suppose  that  T  =  (^ojOj-Pt)  is  a  Turing  machine  over  the  finite  alphabet 
A  =  {tto, . . . ,  Uti},  where  Q  is  the  set  of  states,  Pt  is  the  set  of  commands  of  T, 
qo  is  the  initial  state.  Define  the  following  set  Dt: 

Dt  =  {a  I  q:  is  a  configuration  of  the  Turing  machine  T}. 

The  set  Dt  is  a  finite  automaton  recognizable.  On  the  set  Dt  consider  a  binary 
relation  Rt  defined  by 

Rt  =  {(o!,^)  I  there  exists  a  command  in  Pt  transforming  a  to  /?}. 

The  set  Rt  is  recognizable  by  a  2-variable  automaton.  Define  the  graph 
Qt  =  {Dt,Rt)-  We  get  the  following  proposition. 

Proposition  2.6  The  graph  Qt  =  (Dt^Rt)  is  automatic.n 

Unary  Structures.  We  consider  structures  (A,  /i, . . . ,  fn),  where  each  fi 
is  a  unary  operations  on  A.  Here  are  two  results  on  automatic  presentations  of 
unary  structures. 

Proposition  2.7  Any  free  unary  structure  A  has  an  automatic  presentation. 

Proof.  We  introduce  the  set  X  =  {0,00,000, .. .,0”, .. .}.  Let  S  be 
{0,  fly. , . ,  /77}.  Define  the  following  set  D  over  this  alphabet: 

D  =  {0”a|n>l&:aG{/i,...,/,7}*}. 

The  set  D  is  a  finite  automaton  recognizable.  Each  defines  a  unary  operation, 
also  denoted  by  /j,  by  letting  fi{0^a)  =  O'^afi.  It  is  clear  that  (D,  fi,...ifn) 
is  the  free  unary  algebra  with  the  set  of  generators  X.  By  the  definition  of  /,, 
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we  conclude  that  /*  is  recognizable  by  a  2-variable  automaton.  This  proves  the 
proposition.  □ 

A  unary  structure  (A, /i, . . .  ,/„)  is  abelian  if  for  all  a  6  A,  <  n,  we 
have  fifjia)  =  fjMa). 

Proposition  2.8  Any  free  abelian  unary  structure  has  an  automatic  presenta¬ 
tion. 

Proof.  We  introduce  the  following  alphabet  S  =  {0,ai, . . .  ,an}.  Define 
the  set  D  =  . . .^n  I  >  !}•  The  set  D  is  finite  automaton 

recognizable.  For  each  i  <  n,  define  a  unary  operation  /»  on  the  set  D  as  follows: 
fi{x)  =  ^ if  and  only  if  if  a;  =  ^ . . .  a”* . . .  . 

It  is  easy  to  see  that  is  recognizable  by  a  2-variable  automaton.  There¬ 
fore  the  unary  structure  (D,  /i, . . . ,  /n)  is  automatic.  This  structure  is  the  free 
abelian  unary  structure  on  the  set  {0^  |  5  >  1}  of  generators. □ 

A  permutation  structure  is  a  unary  structure  A  =  (A,/),  where  /  is  a 
1-1  function  defined  on  the  set  A. 


Proposition  2.9  If  the  length  of  finite  cycles  of  f  is  hounded,  then  the  permu¬ 
tation  structure  A  =  (A,  /)  has  an  automatic  presentation. 

Proof.  Suppose  we  are  given  a  permutation  structure  A  =  (A,  /)  which 
satisfies  the  conditions  of  the  proposition.  First,  suppose  that  /  does  not  have 
any  cycles  of  finite  length.  Consider  the  alphabet  {0, 1,2}.  Define  the  following 
set  D  which  is  finite  automaton  recognizable: 


D  =  {1"0*=  I  n,fc  >  1}  U  {0"'2*  I  m  >  l,t  >  0}. 
Define  on  this  set  unary  operation  f  as  follows. 


jn-lQfc 

0m2t+i 


if  a:  =  1^0*  and  n  >  1, 
if  z  =  0”^2^  and  t  >  0. 


The  function  f  is  recognizable  by  a  2-variable  automaton.  Thus,  (D,f) 
is  a  permuation  structure  which  has  infinitely  many  infinite  cycles.  Moreover 
{D,  f)  does  not  have  cycles  of  finite  length.  From  this  we  conclude  that  any  per¬ 
mutation  structure  which  does  not  have  cycles  of  finite  length  has  an  automatic 
presentation. 

Next,  suppose  that  the  length  each  cycle  of  A  =  (A,  /)  is  n.  Consider  the 
set  D  C  {0, 1}*  defined  by  D  =  {0”^1*  |  m  >  l,i  =  {1,2, ...  ,n  -  1}}.  The  set 
D  is  a  finite  automaton  recognizable.  Define  f  as  follows: 


r  if j;  =  0"^randi<n-l 


The  function  is  recognizable  by  a  2-variable  automaton.  Thus  (D,/')  is  a 
permuation  structure  which  is  isomorphic  to  A. 


376 


Now  consider  the  general  case.  Let  (ni, A;i), . . . ,  (nm.km)  be  the  sequence 
of  all  pairs  such  that  for  each  i  <  m,  Ui^ki  <  lj,  and  the  permutation  structure 
A  =  (A,/)  has  exactly  ki  cycles  of  length  Ui.  Combining  the  previous  cases,  we 
can  conclude  that  A  has  an  automatic  presentation. □ 

Is  the  hypothesis  of  the  previous  proposition  necessary?  Here  is  an  example 
which  shows  that  this  is  not  a  case. 

Proposition  2.10  There  exists  an  automaton  presentable  permutation  struc¬ 
ture  such  that  the  set  of  lengths  of  finite  cycles  of  this  structure  is  not  bounded. 

Proof.  Define  the  following  function  /  on  set  {0,1}*.  If  a  =  1”,  then 
/(a)  =  0”.  Suppose  that  a  =  y^Ol",  where  ti  >  1.  Then  /(a)  =  ^10^.  Suppose 
that  a  =  pO.  Then  f{a)  =  pi.  One  can  check  that  /  is  recognizable  by  a 
2-variable  automaton.  Note  that,  for  each  n  the  function  forms  a  cycle  of  length 
2”.D 

Vector  Spaces  and  Abelian  Groups.  First  consider  the  simplest  infinite 
abelian  group,  that  is,  the  rank-one  free  abelian  group  of  rational  integers  (Z,  -f-). 

Lemma  2.1  The  group  (Z, +)  has  an  automatic  presentation. 

Proof.  Each  integer  n  6  Z  is  a  word  over  the  alphabet  E  =  {0, 1, . . . ,  9,  -  }. 
The  standard  algorithm  which  adds  two  integers  gives  a  3-variable  automaton 
over  S  recognizing  the  relation  {(x,^/,  z)\  x  -\-y  =  z}.n 

It  can  be  proved  that  the  direct  product  of  any  two  automata  presentable 
groups  is  also  automaton  presentable.  Since  any  finitely  generated  abelian  group 
can  be  written  as  a  direct  product  of  a  finite  group  and  finitely  many  copies  of 
(Z, -h),  we  get  the  following  proposition. 

Proposition  2.11  Any  finitely  generated  abelian  group  has  an  automatic  pre¬ 
sentation.  □ 

Remark.  Any  finitely  generated  abelian  A  =  (A,  +)  group  with  the  gener¬ 
ators  pi, . . .  induces  a  unary  structure  (A,  /i, . . . ,  /„),  where  fi{a)  =  a gi. 
In  [6]  it  is  proved  that  (A, /i, . . . ,/„)  has  an  automatic  presentation.  That  is, 
any  finitely  generated  abelian  group  is  automatic  in  sense  of  Epstein,  Cannon, 
and  Thurston.  The  proposition  above  is  clearly  a  stronger  version  of  this  result. 

Let  V  =  (V,  0)  be  a  vector  space  over  field  T.  Each  f  induces  a  unary 
operation,  also  denoted  by  /,  on  the  set  Y  by  setting  f(v)  =  vf.  Thus,  we  can 
identify  the  vector  space  V  with  the  structure  (V,  0,  f)feF‘ 

Proposition  2.12  Any  countable  vector  space  (V,  0,  f)feF  over  a  finite  field  T 
has  an  automatic  presentation. 

Proof.  Since  .F  is  a  finite  field,  we  can  suppose  that 
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where  p  is  a  prime  number.  Consider  the  set  D  =  {0, 1, . . .  ,p— 1}*\  {e},  where 
e  is  the  empty  word.  Define  operation  0  on  this  set 


f  il  +  ji  +  jm  if  n  <  m 
\  if  m  <  Tl 


For  each  fc  €  {0, 1, . . .  ,p  -  1}  define  unary  operation  fk  by 


It  can  be  seen  that  the  relations  corresponding  to  the  operations  0  and  fk  are 
recognizable  by  3  and  2-variable  automata,  respectively.  □ 

3  A  Characterization  of  Automatic  Structures 


Suppose  that  (D,  . . . ,  is  a  relational  structure  such  that  D  C  S*, 

P^^  C  ,  where  j  =  1, . . . ,  s,  and  S  is  a  finite  alphabet.  In  this  section  we 

present  an  answer  to  the  following  question.  When  is  this  structure  automatic? 
In  order  to  answer  to  this  question,  we  refine  the  Myhill-Nerode  theorem  which 
characterizes  finite  automaton  recognizable  languages.  It  suffices  to  characterize 
relations  of  arity  n  recognizable  by  n- variable  automata. 

Suppose  that  i?  is  a  relation  of  arity  n.  Let  R*  be  the  language  over  alpha¬ 
bet  (S  U  {O})”  obtained  from  R  by  convolution.  By  the  definition  of  automatic 
structure,  structure  (D,il)  is  automatic  if  and  only  if  D  is  finite  automaton 
recognizable  and  the  convolution  R*  is  recognizable  by  a  finite  automaton  over 
alphabet  (S  U  {O})”.  Thus  the  Myhill-Nerode  theorem  can  be  applied  to  char¬ 
acterize  when  structure  {D^R)  is  automatic.  But  the  finite  automaton  recog¬ 
nizable  languages  obtained  by  the  convolution  of  the  relations  of  arity  n  is  a 
proper  subclass  of  all  finite  automata  recognizable  languages  over  S  U  {O})"^. 
The  original  Myhill-Nerode  theorem  does  not  give  a  characterization  of  this 
class.  But  below,  using  the  idea  behind  the  Myhill-Nerode  theorem  combined 
with  many-sorted  algebra,  we  give  a  self-contained  characterization  of  automa¬ 
ton  recognizable  relations  on  S*.  We  remark  that  there  have  been  deep  and 
interesting  investigations  to  characterize  this  class  of  relations  [19].  Apparently 
our  characterization  of  this  class  of  such  relations  is  different  from  previous  ones, 
possibly  clearer  and  simpler. 

Let  S  be  a  finite  alphabet.  Let  <  be  a  pre-partial  ordering  (reflexive  and 
transitive  binary  relation)  on  E.  We  say  that  elements  <7i,o'2  have  the  same 
sort  if  (Tl  <  (T2  and  a2  <  Thus,  the  elements  of  E  are  sorted.  Moreover, 
since  <  is  a  pre-partial  ordering,  E  is  a  finite  disjoint  union 

El  U  Ea  U  ...  U  Ejfe, 

where  each  E^  contains  all  elements  of  E  of  the  same  sort.  This  induces  a  partial 
order  on 
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If  a  6  Si,  then  we  say  that  (t  has  sort  and  we  denote  it  by  5(a).  If  ai  6  S*, 
<72  €  S_,-  and  Sj  <  Sj,  we  say  that  the  sort  i  is  weaker  then  the  sort  j.  Introduce 
the  following  system  S<: 

(S,<,Ei,. 

Let  El  be  the  smallest  element  with  respect  to  <.  We  call  such  a  system  a 
many-sorted  finite  alphabet. 

Let  S<  be  a  many-sorted  finite  alphabet.  Define  the  set  S<  of  sorted 
finite  words  as  follows.  The  word  cri . . .  (7„  belongs  to  E<  if,  and  only  if,  for 
each  i  <  n  —  1,  the  sort  of  ai  is  weaker  than  the  sort  of  (Ti+i.  We  take  it  that 
the  empty  word  has  the  weakest  sort.  Since  the  alphabet  E  is  sorted  we  can  also 
sort  the  words  in  E<.  The  sort  of  a  word  a  G  E<  is  the  sort  of  the  last  symbol 
appearing  in  a.  Let  Si  be  the  set  of  all  words  orsort  i.  We  call  any  subset  of 
E<  a  many-sorted  language.  We  introduce  the  many  sorted  algebra 

=  (5i,  52, . . . ,  5fe, 

where  the  unary  operation  is  defined  on  a  and  equal  to  acr  if  and  only  if  the 
sort  of  a  is  weaker  than  the  sort  of  a.  We  can  also  define  the  many-sorted 
semigroup 


(5i,52,...,5fc,-), 

where  the  binary  operation  •  is  defined  as  follows.  Let  a,^  G  E<.  Then  a  ■  ^  is 
defined  and  equal  to  a/3  if  and  only  if  ap  G  E<,  that  is,  the  sort  of  a  is  weaker 
than  the  sort  of  the  first  letter  of  p. 

Definition  3.5  A  many-sorted  (non-deterministic)  finite  automaton  over 
the  alphabet  E<  is  a  system  (Qi, . . .  A, F),  is  defined  as  follows. 

1.  Each  Qi  is  the  finite  set  of  states  of  sort  i  and  I  C  Qi  is  the  set  of 
initial  states, 

2.  For  all  distinct  i,  j,  Qi  fl  Qj  =  0. 

S.  F  C  Qi  U  ...  U  Qk  is  the  set  0/ final  states, 

A  C  (Qi  U  . . .  U  Qk)  X  E  X  (Qi  U  . ...  U  Qk)  is  the  transition  table  with 
the  following  property.  If{q,(T,q')  G  A,  then  s(q)  <  s((r)  <  s(q'). 

If  the  condition  (g,<7,qf')  G  A,  (^,<7,g")  G  A  implies  that  q'  =  5",  then  ft  is 
called  deterministic. 

We  can  define  the  notion  of  computation  of  many  sorted  finite  automata  on 
sorted  words.  For  a  many  sorted  finite  automaton  D,  we  can  define  the  set  L(n) 
of  all  sorted  words  accepted  by  the  automaton.  By  the  definition  of  many-sorted 
automaton,  a  word  accepted  by  the  many-sorted  automaton  must  belong  to  E<. 
We  call  the  set  L{Q)  recognizable  by  the  many-sorted  automaton.  The 
following  lemma  can  be  proved  using  the  standard  methods  of  finite  automata 
theory. 
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Lemma  3.2  The  set  of  many-sorted  finite  automaton  recognizable  sets  is  closed 
under  intersection  and  union.D 

Though  the  next  lemma  also  uses  the  known  methods  of  finite  automata 
theory,  we  present  a  brief  proof  of  the  lemma. 

Lemma  3.3  For  any  many-sorted  finite  automaton  Cli,  there  exists  a  deter¬ 
ministic  many-sorted  finite  automaton  accepting  the  same  language  accepted  by 
fli . 


Proof.  Let  Cli  =  (Qi, . . .  A,F).  Define  the  following  many-sorted 

finite  automaton  ^2: 

1.  For  each  i  the  states  of  sort  i  are  the  subsets  of  Qi. 

2.  The  set  of  initial  states  contatins  only  one  element  which  is  /. 

3.  The  set  of  final  states  consists  of  all  subsets  intersecting  F. 

4.  The  transition  table  contains  all  such  triples  (Q,  cr,  Q')  such  that  Q  and  Q' 
are  the  states  of  the  new  automaton  and  the  following  holds: 

(a)  For  any  q  G  Q  there  exists  q'  G  Q'  for  which  {q^o-^q*)  G  A, 

(b)  For  any  q'  G  Q'  there  exists  ^  G  0  for  which  {q,  cr,  q')  G  A. 

Then  ^2  is  a  many-sorted  deterministic  finite  automaton  and  accepts  exactly 
those  words  accepted  by  the  original  automaton  Hi.D 

Lemma  3.4  For  any  many-sorted  finite  automaton  Hi,  there  is  a  many-sorted 
finite  automaton  accepting  the  language  E<  \L{Cli). 

Proof.  By  the  previous  lemma  we  may  assume  that  Qi  =  {Qi , . . . ,  A,  F) 
is  deterministic.  Thus  ^2  =  (Qi, •  •  •  accepts  the  complement  of 

L(Di).n 

As  a  corollary  of  the  previous  lemmas  we  get  the  following  theorem. 

Theorem  3.1  The  set  of  all  many-sorted  finite  automata  recognizable  langauges 
0/  E<  forms  Boolean  algebra. D 

Let  F  =  (5i,  5'2, . . . ,  Skif(T)<T^E  be  the  above  defined  many-sorted  algebra. 
An  equivalence  realtion  7/  C  E<  is  called  a  congruence  if  it  satisfies  the  following 
conditions: 

1.  For  all  G  77,  the  words  a  and  have  the  same  sort. 

2.  For  all  cr  G  S  and  G  rj,  if  fa{oi)  is  defined,  then 
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If  t;  is  a  congruence  relation  on  the  many-sorted  algebra  then  i)  is  also  a 
right  congruence  relation  of  the  many-sorted  semigroup  (^i, . . . ,  •).  That 

is,  7}  satisfies  the  following  condition:  for  all  (a,/3)  G  rj  and  u  G  if  a  *  u  is 
defined,  then  (aujfiu)  G  rj. 

Let  L  be  a  many-sorted  language  over  S<.  Define  the  equivalnce  relation 
r]i  as  follows.  For  all  a,  /3  G  S<,  a  and  P  are  ??L“equivalent  if 

1.  a  and  ^  have  the  same  sort,  and 

2.  for  all  u  G  a  •  u  G  L  if  and  only  if  /?  •  u  G  L. 

Then  is  a  congruence  relation  on  the  algebra  T  (equivalently,  is  a  right 
congruence  relation  of  the  many-sorted  semigroup  (E<,*)).  If  the  alphabet 
has  only  one  sort,  this  equivalence  is  Myhill-Nerode  equivalence.  The  version 
of  the  Myhill-Nerode  theorem  below  gives  a  characterization  of  many-sorted 
automata  recognizable  languages. 

Theorem  3.2  Lei  L  he  a  many-sorted  language  over  the  alphabet  S<.  The 
following  conditions  are  equivalent: 

1.  The  language  L  is  recognizable  by  a  many-sorted  finite  automaton. 

2.  The  langauge  L  is  a  union  of  some  equivalence  classes  of  a  right  congruence 
relation  rj  of  finite  index. 

Outline  of  Proof.  Let  be  a  many-sorted  automaton  accepting  L.  We 
may  suppose  that  ft  is  deterministic.  Then  the  equivalence  relation  tjl  has  finite 
index  and  is  a  right  congruence  relation.  Suppose  that  L  is  the  union  of  some 
equivalence  classes  of  a  right  congruence  relation  7)  of  finite  index.  Define  a 
finite  many-sorted  automaton  accepting  L  as  follows.  The  initial  state  of  the 
automaton  will  be  the  empty  word.  The  states  of  sort  i  are  the  7^— equivalence 
classes  of  words  of  sort  i.  A  state  g  is  a  final  state  of  the  automaton  if  g  is  a 
subset  of  L.  A  pair  {q^cT^q')  belongs  to  the  transition  table  of  the  automaton  if 
q  •  a  belongs  to  the  r/— equivalence  class  q'.  One  can  verify  that  this  automaton 
is  many-sorted  and  accepts  L.D 

We  apply  this  theorem  to  characterize  automatic  structures.  Suppose  that 
A  is  a  finite  alphabet  and  that  O  ^  A.  Introduce  the  alphabet 

S  =  (A  U  {0})”\{0}^ 

Define  a  binary  relation  <  on  the  alphabet  S  as  follows: 

if,  and  only  if,  for  all  i  G  {1, . . . ,  n},  the  condition  (ti  —  O  implies  that  =  O. 
It  follows  that  <  is  a  pre-part ial  ordering  of  the  alphabet  E,  giving  us  a  many- 
sorted  finite  alphabet  E<.  By  the  definition  of  the  convolution  operation,  the 
convolution  R*  of  any  relation  R  of  arity  n  is  a  many-sorted  language  over  E< . 
The  previous  result  implies: 
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Theorem  3.3  Let  A  be  a  finite  alphabet,  and  let  {D, . . . ,  be  a  struc¬ 
ture  with  D  C  A  and  P/”*  C  (A*)'^%  where  iO, . . . ,  5.  For  each  i  €  {0, . . . ,  s} 
consider  the  many-sorted  alphabet  Il<,i  =  (A  U  {O})'’*  \  Then  the 

following  statements  are  equivalent: 

1.  The  structure  {D,  P^° , . . . ,  PJ”-)  is  automatic. 

2.  The  set  D  is  a  finite  automaton  recognizable  and  the  convolution  of  each 
set  P/*  is  recognizable  by  a  many-sorted  finite  automaton. 

3.  The  set  D  is  a  finite  automaton  recognizable  and  the  convolution  of  each  set 
P/*  is  a  union  of  some  equivalence  classes  of  a  right  congruence  relation 
7}  of  finite  index  on  the  many-sorted  semigroup  ((Si, . . . ,  5*,  •).□ 

We  apply  the  discussion  above  to  characterize  automatic  structures  over 
one  letter  alphabet  S  =  {!}.  We  identify  the  set  S*  with  the  set  uj.  For  clarity 
consider  structures  with  domain  u  and  binary  relations,  that  is,  structures  of 
type 

(a;,Po,...,P,), 

where  each  Pi  is  a  binary  relation.  We  introduce  the  3-sorted  alphabet 

E<  =  {(1,1),  (1,0),  (0,1)}, 
where  (1,1)  <  (1,0)  and  (1,1)  <  (0,1).  Let 

n  =  (Qi,Q2,03,^o,  A,P) 

be  a  3-sorted  finite  deterministic  automaton  over  the  alphabet  described  as 
follows.  The  set  Qi  can  be  thought  as  a  graph  which  forms  a  loop.  All  transitions 
in  Qi  are  labelled  by  (1,1).  Each  s  e  Qi  forms  two  disjoint  loops  and 

.  All  transitions  in  are  labelled  by  (1,0)  and  all  transitions  in 

are  labelled  by  (0, 1).  We  also  can  assume  that  if  s,  s'  €  Qi  are  different 
states,  then  the  loops  are  disjoint. 

A  set  M  C  ci;  is  an  arithmetic  progression  if  there  exist  numbers  ni  < 
. . .  <  njfe  e  (J  such  that 


M  =  {ni,...,nk}  U  {ukt  \teu)}. 

A  subset  of  uj  is  automatic  if  it  is  a  finite  union  of  arithmetic  progressions.  It 
is  easy  to  see  that  a  set  is  automatic  if  and  only  if  it  is  recognizable  by  a  finite 
automaton  over  (1). 

Corollary  3.1  Let  (a;,Po, . . . ,P«)  be  a  structure  such  that  each  Pi  is  a  binary 
relation.  This  structure  is  automatic  if  and  only  if  for  each  f  €  {0, . . .  ,7i}  there 
exist  automatic  sets 

Ai ,  Pi , .  .  .  ,  Aij^  ,  Pij,  ,  Cl ,  Di ,  • .  •  ,  Ci^  ,  Pit .  j 
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such  that 

ti 

Pi  =  +  I  ®  ^  ^  ^  |j{(a:  +  n,»)  |  n  e  C,,a;  e  !>,}.□ 

i=i  «=i 

4  Basic  Properties  of  Automatic  Presentations 

Features  of  Decidability.  Investigation  of  automaton  recognizable  relations 
over  E*  suggests  investigating  the  corresponding  predicate  calculus.  Thus,  if 
Ri  and  R2  are  automaton  recognizable,  one  can  define  relations  corresponding 
to  the  expressions  (i?i  V  R2),  {Ri  A  R2),  and  ->(i?i),  3xRi,  and  For 

instance,  suppose  Ri  is  an  n-ary  relation.  Define 

3a;i(i?i)  =  {(xi, . . . , I 

{x\ , . . . ,  Xi—i ,  ajj,  ajj'-j-i , . . . ,  Xyi)  ^  Ri } , 

V®i(-Ri)  —  {(3^1 )  •  •  •  j ,  ajj-i-i , . . . ,  ajyj)  I 

Vq:  R(^Xi , . . . ,  1 )  o,  ,  • .  • ,  a^Ti)}. 

When  Ri  is  a  unary  relation,  then  corresponds  to  0  if  JRi  =  0,  and  to  E* 

otherwise.  Similarly,  'ix(Ri)  corresponds  to  0  if  jRi  ^  E*,  and  to  E*  otherwise. 

The  next  theorem  is  a  consequence  of  standard  finite  automata  theory  of 
the  middle  1950’s.. 

Theorem  4.4  1.  Let  Ri  and  R2  be  automata  recognizable  relations.  Then 

the  relations  corresponding  to  the  expressions  {R1VR2),  {R1AR2),  -'(Ri), 
3a;(jRi),  and'ix{Ri)  are  also  automata  recognizable. 

2.  The  emptiness  problem  for  n -variable  automata  is  decidable  uniformly 
in  n. 

3.  There  exists  a  procedure  which,  for  automata  recognizing  Ri  and  R2,  con¬ 
structs  automata  for  recognizing  the  relations  corresponding  to  the  expres¬ 
sions  {Ri  y  R2),  (Ri  AR2),  ~'{Ri),  3x{Ri),  andyx{Ri).  □ 

This  theorem  implies  several  properties  of  automatic  presentations. 

Corollary  4.2  Suppose  that  a  structure  A  has  an  automatic  presentation.  Then 

1.  There  exists  an  effective  procedure  which,  applied  to  a  first  order  definition 
of  a  relation  P  on  A,  yields  an  algorithm  deciding  P. 

2.  The  first  order  theory  of  the  structure  A  is  decidable. 

Proof.  Let  1/  :  D  A  he  an  automatic  presentation.  Then  the  atomic 
relations  and  the  equality  relation  on  this  structure  are  decidable  under  the 
presentation  i/.  By  the  theorem  just  stg-ted,  there  exists  an  effective  procedure 
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which,  for  any  first  order  definable  relation  P,  produces  an  algorithm  deciding 
P.  It  also  follows  that  the  first  order  theory  of  this  structure  is  decidable. □ 

Remark.  In  fact,  if  A  is  automatic,  the  corollary  above  can  be  strengthened. 
Namely,  consider  the  set  of  all  polynomials  over  the  structure  A.  (This  is  the 
set  of  all  functions  of  the  form 

t(ai, . , . , a/t+i, . . . ,  am),  where  a:  is  a  variable,  T  is  a  term,  and  ai, . - . , 
are  elements  of  A.)  Using  the  decidability  of  the  monadic  second  order  theory 
of  two  successor  functions  [12],  it  follows  that  the  first  order  theory  of  A  plus 
the  monadic  second  order  theory  of  polynomials  over  A  is  decidable  [20]. 


Corollary  4.3  If  structure 


has  an  automatic  presentation,  then  there  exists  an  automatic  presentation  p,  : 
a  which  is  a  1-1  function. 


Proof.  Let  i/  :  Pi  A  be  an  automatic  presentation  of  A.  Let  D  C 
{0, 1, . . . ,  71  -  1}*.  Define  ordering  :<  on  set  S*  as  follows.  If  the  length  of  a  is 
less  than  the  length  of  then  a  <  If  a  and  /?  have  the  same  length,  then 
a  :<  P  \i  and  only  if  there  exist  7,71,72  €  S*  such  that  a  =  7771,  =  7^72 

and  z  <  j  <  n  ~  1.  This  relation  is  recognizable  by  a  2-variable  automaton. 
Moreover  is  a  linear  ordering  of  S*  isomorphic  to  the  natural  ordering  of  uj. 
Consider  the  following  relation 

D  =  {a\ae  Dk^l3{i/{a)  =  i/{P)  a  r<  ^}. 

By  the  theorem  above,  the  set  P  is  a  finite  automaton  recognizable.  For  every 
i  <k  there  exists  a  nj-variable  automaton  which  recognize  the  set 


By  the  theorem  above,  the  set 

A  t=  ,  I'CanJ,  i^(a„..+i))} 

is  also  recognizable  by  an  (7ii+l)-variable  automaton.  Similarly,  for  each  j  <  s, 
the  set 


is  recognizable  by  an  ruj-variable  automaton.  It  follows  that  the  structure 


is  isomorphic  to  A.  The  mapping  p:  D  A  defined  by  p{a)  =  u{a)  is  1-1.  □ 
Using  the  theorem  above,  similar  to  the  previous  corollary,  one  can  prove 
that  automata  presentable  structures  are  closed  under  direct  product  and  fac¬ 
torizations  with  respect  to  2- variable  automata  recognizable  congruences. 
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Corollary  4.4  1.  Let  vi  \  Di  Ai  and  U2  :  D2  —*  A2  be  automatic  presen¬ 

tations  of  structures  Ai  and  A2  of  the  same  signature.  Then  the  structure 
Ai  X  A2  possesses  an  automatic  presentation. 

2.  Let  v\  \  D  A  he  an  automatic  presentation  of  Ai.  If  7}  is  a  congruence 
of  A  recognizable  by  a  2~variable  automaton,  then  the  factor  structure  A/rj 
possesses  an  automatic  presentation.  □ 

Finitely  Generated  Automatic  Structures.  Suppose  that  a  structure 
A  is  finitely  generated.  Let  ai,...,ak  be  generators  of  this  structure.  Let 
/^^  •  •  •  j  fr*  all  atomic  operations  of  the  algebra.  We  define  the  following 
sequence  of  finite  sets  generating  the  structure. 

Stage  0.  Put  Go  =  {ao, . . . ,  ajb}. 

Stage  n+l.  Suppose  that  Gt  has  been  defined.  Then 

Gn^l  =  U  . . . ,  6^,)  I  i  =  1, .  .  . ,  ,  hm,  €  G^}. 

Definition  4,6  Consider  the  function  f  defined  by  f{n)  —  card{Gn)-  We  call 
this  function  the  grouth  level  of  the  generators  ai , . . . ,  ajb . 

Lemma  4.5  Let  Ui, . . .  ,ajfe  be  generators  of  the  automatic  structure  A  over  the 
alphabet  S  of  cardinality  s.  Then  there  exist  a,b  £  u  such  that  the  growth  level 
of  the  generators  does  not  exceed 

Proof.  Since  A  is  automatic,  there  exist  finite  automata  rec¬ 

ognizing  the  graphs  of  the  operations.  Let  a  be  the  maximum  of  the  lengths 
of  the  generators  ai,...,a/k.  Let  b  be  an  upper  bound  for  the  number  of 
states  of  all  automata  0^,  i  =  1, , . .  ,n.  We  can  prove  by  induction  on  n  that 
card(Gn)  <  5“+!+^”. 

For  the  base  step,  if  n  =  0,  card{Go)  <  since  the  number  of  elements 
of  length  n  does  not  exceed 

For  the  induction  step,  sppose  the  conclusion  is  true  for  card{Gn)  <  5®+^+*^. 
Let  a  =  &m)  for  some  atomic  operation  /  and  61, . . . ,  6^  G  G„.  By 

inductive  hypothesis,  the  lengths  of  the  bi  do  not  exceed  a  H-  bn.  It  follows  that 
the  length  of  a  does  not  exceed  a  -I-  6(n  -f- 1).  Thus  Gn+i  is  a  subset  of  the  set 
of  all  words  of  length  not  exceeding  <a-\-b{n-\- 1).  The  cardinality  of  this  set 
is  at  most  s<^+^+Hn+i)^  as  required.  □ 

Corollary  4.5  Let  A  he  an  automatic  structure.  For  any  substructure  generated 
by  some  elements  ui, . . . ,  there  exist  s,a,b  ^  u  such  that  the  growth  level  of 
the  generators  does  not  exceed 

Using  this  lemma  one  can  get  many  examples  of  structures  which  do  not  have 
automatic  presentations  but  still  possess  the  features  of  decidability  mentioned 
above. 

Corollary  4.6  Suppose  that  f  is  a  function  symbol  of  arity  2  and  c  is  a  constant 
symbol.  Then  the  absolutely  free  algebra  generated  by  c  and  f  does  not  have  an 
automatic  presentation. 


385 


Proof.  The  elements  of  the  algebra  are  the  terms  defined  by  the  following 
inductive  definition. 

1.  c  is  a  term. 

2.  If  h  and  *2  are  terms,  then  /(ti,t2)  is  a  term. 

Let  T  be  the  set  of  terms.  The  free  algebra  is  (T,  /).  The  generator  of  this  algebra 
is  c.  The  lower  bound  for  the  growth  level  of  the  generator  c  is  2^  —  n.  Therefore 
the  absolutely  free  algebra  (T,  /)  does  not  have  an  automatic  presentation.  □ 

5  Strongly  Automatic  Presentations 

In  this  section  we  give  a  characterization  of  strongly  automata  presentable 
structures.  Let  S  be  a  finite  alphabet,  O  0  S  and  n  >  1.  Put  So  =  S  U  {O}. 
On  the  set  of  we  define  a  relation  •  Let  a,  ^  .  Let  ao  be  the  word 

obtained  from  a  ommiting  all  occurences  of  O.  Define 

a  ^  ^  ao  =  /3o  • 

Thus,  is  an  equivalence  relation  on  .  We  can  then  define  an  equivalence 
relation  on  the  set  of  all  n-tuples  of  EJ,  as  follows.  Two  n-tuples  are 
equivalent  if  and  only  if  their  corresponding  components  are  <^0  -equivalent. 
By  the  definition  of  ,  that  each  -equivalence  class  is  represented  by  some 
unique  n-tuple  from  the  set  (E*)”.  Thus,  there  is  a  natural  1-1  correspondence 
between  the  factor  set  (E^  )”/  and  (E*)^.  Define  a  binary  operation  on 
(E*)”  as  follows. 

(Q!1  , . .  •  ,  0!n)(/^l  j  •  •  •  j  /^n)  “  >  •  •  •  >  ^n/^n) 

This  operation  is  well-defined  on  -equivalence  classes.  Let  be  a  relation 
on  E*  of  arity  n.  Define  a  Myhill-Nerode  equivalence  relation  on  (E*)"^  by: 

OL  '^R  f3  ^  Wu  G  (E*)”(au  E  il  •<->  I3u  E  R) 

Then  ^^r  is  an  equivalence  relation  compatible  with  the  right  multiplication, 
that  is,  if  a  ^r  then  for  all  w,  au  >^r  pu. 

Definition  5.7  An  n-variahle  automaton  D  =  {S,Sq,AjF)  on  E  is  simple  if: 

1.  For  all  s  £  S  and  a  =  (ai, . .  .,an),P  =  •  •  j^n)  ^  (Llo  )”;  P) 

then  A(s,  ai  ★ . . .  ★  a„)  =  A(s,  ★ . . .  ★  ^n)* 

2.  For  all  s  e  S,  a  =  {<ri, . .  •  ,0?^  =  (<5i, .  •  •  ,<5n)  ^  (So  if  A{s,(t)  = 
A(s,(5),  then  A(5,(t)  =  A(5,  (71, . . .  ,7n)),  where £  {au Si}. 

The  next  theorem  characterizes  strongly  recognizable  relations  in  terms  of 
simple  automata,  the  equivalence  and  finite  automata  recognizable  subsets 
of  E*.  The  index  of  an  equivalence  relation  is  the  number  of  equivalence  classes. 
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Theorem  5.5  Let  E  be  an  alphabet  Let  R  be  a  relation  on  E*  of  arity  n.  The 
following  statments  are  equivalent: 

1,  R  is  a  strongly  n~recognizable. 

2.  R  is  a  union  of  some  classes  of  an  equivalence  relation  with  a  finite  index 
and  compatible  with  the  right  multiplication  on  (E*)’^. 

3.  R  is  accepted  by  a  simple  n- variable  automaton  over  E. 

4,  There  exists  k  £  u)  and  finite  automata  recognizable  subsets  Riij,..,Rni 
of  E*,  where  l<i<k,  such  that  R  ~  JRh  x  . . .  x  Rni. 

Proof.  1)  — ►  2).  Let  U  =  (*9,  ^o,  A,F)  be  an  n~variable  strong  automaton 
accepting  R.  Define  a  congruence  relation  ~  on  (E*)”  as  follows: 

(ai,.. . ,Q!n)  -- (A,-.., /5n) 

if,  and  only  if,  for  all  5,^  G  5”  and  for  all  i  <  n,  there  exists  a  computation  of 
Q  on  ai  which  begins  in  state  g  and  ends  in  state  s  if  and  only  if  there  exists 
a  computation  of  Q  on  pi  which  begins  in  q  and  ends  in  5.  The  relation  ^  is  a 
congruence  of  a  finite  index.  Since  R  is  recognizable  by  0,  by  the  definition  of 
i?  is  a  union  of  some  ^-equivalence  classes. 

2)  — >  3).  Let  ^  be  a  given  equivalence  compatible  with  the  right  mul¬ 

tiplication.  Suppose  that  i?  is  a  union  of  some  classes  of  this  equivalence 
relation.  Define  an  n- variable  automaton  ft.  Let  the  set  of  states  to  be 
S  I  Oi  £  (E*)”},  where  aZ  is  the  -closure  of  the  '^-equivalence  class 

containing  a  £  (E*)”;  let  the  set  of  initial  states  to  be  5o  =  {(A, . . . ,  A)^  }, 
where  A  is  the  empty  word;  put  F  =  \  a  £  R};  define  the  transition 

table  A  as  follows:  for  all  aZ  and  a  £  (Eo  )”  put  A(a^  ,  (t)  =  ioia)Z  .  This 
automaton  is  simple. 

3)  — +  4).  Suppose  that  R  is  accepted  by  a  simple  n-variable  automaton 
A  over  E.  Let  /i,...,/*  be  all  final  states  of  the  automaton  A.  Let  R{fi) 
be  the  set  of  all  n-tuples  which  transform  the  initial  state  go  of  A  to  /».  Let 
^(/i>  1)>  •  •  •  >  Rifii  be  the  projections  of  R{fi)  onto  corresponding  components. 
Using  the  definition  of  simple  automaton,  we  get  R{fi)  =  R{fi,  1)  x . . .  x  R{fi,  n). 
Thus, 

k 

R=\jR{ful)^...xR{U,n). 

i-1 

Note  that  for  all  i,j  the  set  R{fiJ)  is  a  finite  automaton  recognizable  subset  of 
E*. 

4)  — ♦  1).  For  any  1  <  t  <  fc,  there  exists  a  finite  automaton 

^^^  =  (5'^,g^o,Ai,Fil  U  .,.U  Fin) 

such  that  for  any  a  £  E*,  a  £  Rij  if  and  only  if  there  exists  a  computation 
of  Qi  which  begins  in  qio  and  ends  in  Fij.  We  can  suppose  that  for  alH  ^ 
Sif]Sj  =  0.  Define  an  automaton  D  =  (5, 5*0,  A, F)  as  follows. 
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1.  5  =  5i  U  . . .  U  5„  and  5o  =  {gio*,  •  •  • ,  to}- 

2.  F  —  Ui=0  X  ...  X  Fin- 

3.  A  =  UAi. 

This  automaton  is  an  n-variable  strong  automaton  which  accepts  R.  □ 

This  theorem  and  the  theorem  of  the  previous  section  allow  us  to  obtain  a 
corollary  characterizing  structures  which  possess  strongly  automatic  presenta¬ 
tions.  We  need  the  following  notion. 

An  n-ary  relation  P  on  a  set  A  is  called  complete  if  there  exist  subsets 
Ail , ,  ^in  of  A  and  a  number  k  euf  such  that  P  =  Ui=i  Aji  x  . . .  x  Ain- 

Theorem  5.6  A  structure 


A  =  (A;Po”^°,...,PD 

has  a  strongly  automatic  presentation  if  and  only  if  each  predicate  Pf*  is  com¬ 
plete. 

Proof.  For  simplicity  suppose  that  t  =  0  and  mo  =  n.  If  A  has  a  strongly 
automatic  presentation  {L,R)  then,  by  the  previous  theorem,  there  are  recog¬ 
nizable  sets  Rii,  -  -  -  )Rni  such  that  R  =  ^  Pm--  Hence  Pq  °  is  a 

complete  relation. 

Conversely,  suppose  that  P  is  a  complete  relation  on  A.  Then  there  exist 
subsets  Ail, . . . ,  Ain  and  a  number  k  e  u  such  that  P  =  [J.  An  x  ...  x  Ain- 
Consider  a  structure  Ai  =  (A;  An , . . . ,  Ain,  •  •  •  >  ^ki ,  -  •  •  ?  A^n)-  Structure  A  is 
isomorphic  to 

(Aj  Ail  X  ...  X  Ain)- 
i 

By  Proposition  2.1,  Ai  has  a  strong  automatic  presentation 
(L;  Pll,  •  •  •  ,  Pin,  •  •  - ,  Pfel,  -  •  - }  Pfcn)* 

Let  P  =  Ui  Pti  X  •  •  •  X  ^in-  By  the  previous  theorem,  relation  P  is  accepted  by 
an  n-variable  strong  automaton.  Thus  the  structure  (L;  P)  is  isomorphic  to  A- 
Hence  A  possesses  a  strongly  automatic  presentation. □ 

Corollary  5.7  The  structures  (cj;  <)  and  {u;s)  do  not  possess  strongly  auto¬ 
matic  presentations.O 


6  Automatic  Isomorphism  Types 

A  basic  problem  for  recursive  and  polynomial  time  structures  is  to  charac¬ 
terize  structures  which  have  the  same  isomorphism  type  via  recursive  or  p-time 
computable  isomorphisms.  A  structure  is  recursively  (polynomial  time)  categor¬ 
ical  if  any  two  recursive  (p-time)  presentations  of  the  structure  are  recursively 
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(p-time)  isomorphic.  Thus  in  some  sense  a  recursively  (p-time)  categorical 
structure  is  one  for  which  the  problem  of  recursive  (p~time)  presentations  has  a 
unique  solution.  It  looks  very  hard  to  find  general  necessary  and  sufficient  con¬ 
ditions  for  structures  to  be  recursively  (p-time)  categorical.  The  corresponding 
problem  for  automatic  isomorphism  types  is  easy. 

Definition  6.8  We  say  that  sets  Ri,R2  C  S*  have  the  same  automatic  iso¬ 
morphism  type  if  there  exists  a  relation  /,  recognizable  by  a  2-variable  au¬ 
tomaton,  such  that  dom(f)  =  Ri,  range{f)  =  R2,  and  f  is  a  1-1  function. 
In  this  case  we  say  that  Ri  and  R2  are  automatically  isomorphic  via  the 
automatic  isomorphism  /. 

For  an  R  we  denote  by  AI{R)  the  class  of  all  sets  automatically  isomorphic  to  R. 
Note  that  if  L  €  AI{R),  then  L  is  a  finite  automaton  recognizable.  It  is  obvious 
that  if  is  a  finite  set,  then  R2  €  AI{Ri)  if  and  only  if  card{Ri)  =  card{R2). 
Our  next  result  shows  that  for  any  infinite  recognizable  set  there  exists  in  some 
sense  a  standard  presentation  of  this  set  which  has  the  same  AI  type.  We  need 
definitions. 

Definition  6.9  A  set  A  C  S*  is  a  free  monoid  if 

1.  There  exists  a  B  C  A  such  that  B*  =  A,  and 

2.  for  all  bi, ...  ,ba,ai, ...  ,ak  €  B  if  bi ..  .bg  =  ai . . . then  k  —  n  and 
Ui  =  bi  for  alli  =  l,...,n. 

Let  A,B  C  S*.  The  mulitiplication  A  •  B  is  free  if  for  for  all  ai,a2  E 
A,bi,b2  B  if  tti  •  61  =  02  •  62>  then  oi  =  02  and  61  =  62* 

Using  Eilenberg’s  decomposition  theorem  for  finite  automata  recognizable 
sets  [5],  one  can  characterize  automatic  isomorphism  types  for  structures  in  the 
language  of  pure  equality. 

Theorem  6.7  Let  L  be  a  finite  automaton  recognizable  infinite  set  There  exist 
finite  automata  recognizable  pairwise  disjoint  sets  Ln,  ...,  Likx, 

Lfikn  such  thati 

1.  For  all  i,j ,  i  ~  1, . . .  ,n,  j  =  1, . . .  ,ki,  the  set  Lij  is  a  free  submonoid. 

2.  For  all  i  ^  j,  i  <  n,  the  multiplication  Ln  ‘  ...•  Liki  is  free. 

3.  L=  {Lii  ■ . . .  •  Likx )  U  . . .  U  (L„i  • . . .  •  Lkrik )  €  l^iR),  where  AI{R)  is  the 
automatic  isomorphism  type  of  R. 

That  is,  every  finite  automata  recognizable  set  is  automatically  isomorphic  to  a 
finite  union  of  free  mulitiplications  of  free  monoids.  D 

Now  we  investigate  the  more  general  problem  of  automatic  isomorphism 
types  of  automatic  structures. 
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Definition  6.10  Let  A  he  an  automatic  structure.  An  automatic  structure  B  is 
automatically  isomorphic  to  A  if  there  exists  a  function  /,  recognizable  by  a 
2-variable  automaton,  such  that  dom{f)  =  A,  range{f)  =  B  and  f  induces  an 
isomorphism  between  these  structures. 

Let  A  be  automaton  presentable  structure.  The  number  of  automatic  iso¬ 
morphism  types  of  A  we  call  the  automatic  dimension  of  A.  The  structure 
A  is  automatically  categoricalif  its  automatic  dimension  is  1. 

Theorem  6,8  The  automatic  dimension  of  any  automaton  presentable  struc¬ 
ture  is  either  (jj  or  1.  Moreover,  such  a  structure  is  automatically  categorical  if 
and  only  if  its  domain  is  finite. 

Proof.  Let  B  be  an  automaton  recognizable  set.  Let  us  consider  the  se¬ 
quence  bo  :<  hi  ■<  b2  of  all  elements  of  the  set  B,  where  ■<  is  the  linear 

ordering  on  E*  defined  in  the  proof  of  Corollary  4.2.  We  define  a  function 
fB{n)  =  length{bn). 

Lemma  6.6  Let  B,C  be  automaton  recognizable  sets.  If  the  sequence  {fs^  — 
fc  I  increasing,  then  B  and  C  do  not  have  the  same  automatic  isomorphism 
type. 

Proof.  Suppose  that  the  lemma  is  not  true  and  B,Che  automaton  recogniz¬ 
able  sets  such  that  the  set  {\fB{n)  —  fc{n)\  |  n  6  w}  is  not  bounded.  Let  g  be  1-1 
function  such  that  g{B)  =  C.  Suppose  that  g  is  recognizable  by  a  2-variable  au¬ 
tomaton.  Let  n  be  such  that  for  any  6  6  B,  we  have  \length(b)  —  length{g{h))\  < 
n.  We  may  assume  that  n  is  the  number  of  states  of  the  automaton  recognizing 
g.  Indeed,  otherwise,  applying  the  pumping  lemma  of  finite  automata  theory, 
we  would  contradict  the  fact  that  g  is  1-1.  Since  the  sequence  |/b(^)  -  fc{m)\ 
is  not  bounded,  there  is  an  s  €  cj  such  that  |/b(s)  -  fc{s)\  >  n.  There  are  two 
cases. 

Case  1.  Suppose  that  fc{^)  —  fsis)  >  n.  Then  g{hi)  ^  {cs,Cs+i^ . . .}  for  all 
i  <  s.  Since  g  is  1-1,  we  should  have  {p(6o), . . .  ,p(6«_i)}  —  {co, . . . ,  c,_i}.  But 
we  also  have  g{hm)  6  {co, . . . ,  Cg-i},  a  contradiction. 

Case  2.  Suppose  that  fsis)  -  fc{s)  >  n.  Then  /(6m)  ^  {co, •  • .  for  all 
m>  s.  Thus  g  is  not  1-1,  a  contradiction.  We  proved  the  lemma. 

Let  A  be  an  infinite  automatic  structure.  Let  c  be  a  new  symbol  which  does 
not  belong  to  S.  From  the  structure  A  =  {A;Pq°  ,. . .  ,Pf*)  we  define  a  new 
structure  B„: 

1.  The  universe  of  Bn  is 

Bn  =  {aoc”aic” . . .  ttmc”  I  ao . . .  am  C  A} 

(Notice  if  n  =  0  then  Bn  =  A.) 

2.  For  each  predicate  P^*  we  define  a  predicate  as  follows.  A  tuple 

(aoic^'aiic” . .  .ttmoic”,  • .  •  ,aojfeiC”aifc.c”  . .  .am^fciC”) 
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belongs  to  if  and  only  if  (aoi . . .  ttmoi  j  •  •  • ,  aiki  •  •  •  (^rmki )  belongs  to 

pki 

■*  i  • 

This  defines  a  structure 

Bn  =  {Bn;Q^o° . Qi‘)- 

i,Prom  the  construction,  Bn  is  isomorphic  to  A.  Since  A  is  an  automatic  struc¬ 
ture,  it  follows  that  Bn  is  also  an  automatic  structure. 

The  the  sequence  -  /B„(m)|  is  increasing  for  any  fixed  n  e  uj. 

The  lemma  above  implies  that  the  structure  Bn  is  not  automatically  isomorphic 
to  the  structure  Bn-i-  To  complete  the  proof,  note  that  finite  structures  are 
automatically  categorical.  □ 

7  Conclusion  and  Open  Questions 

The  theory  of  automatic  structures  can  be  considered  as  a  branch  of  the 
theory  of  recursive  structures.  But  one  has  to  take  into  account  the  differences 
between  these  two  approaches  for  investigating  the  connections  between  alge¬ 
braic,  model-theoretic,  and  effective  properties  of  structures.  Recursive  model 
theory  can  be  viewed  as  an  application  of  recursion  theory  to  model  theory,  while 
the  theory  of  automatic  structures  can  be  viewed  as  an  application  of  complexity 
theory  to  model  theory. 

For  example,  suppose  that  we  have  a  structure  A  and  a  relation  R  which 
is  of  particular  interest.  If  we  are  interesting  in  deciding  R,  from  the  recursive 
structures  point  of  view,  we  would  consider  the  following  type  of  questions. 

1.  Does  there  exist  a  recursive  copy  of  A  on  which  R  is  decidable? 

2.  Does  there  exists  a  recursive  copy  of  A  on  which  R  is  creative  set? 

3.  Does  there  exist  a  recursive  copy  of  A  on  which  JR  is  a  simple  set,  or  'has 
a  particular  Turing  or  m-degree? 

But  from  the  point  of  view  of  automatic  structures,  we  would  naturally 
consider  the  following  questions. 

1.  Does  there  exist  an  automatic  copy  of  A7 

2.  Does  there  exist  a  automatic  presentation  of  A  on  which  R  is  decidable? 

3.  Does  there  exist  an  automatic  presentation  of  A  in  which  there  is  a  decision 
procedure  for  R  of  given  time  or  space  complexity? 

4.  Does  there  exist  an  automatic  presentation  of  A  in  which  the  decision 
procedure  for  R  is  iVP-complete? 
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This  paper  suggests  that  recursive,  algebraic,  model  theoretic,  and  com¬ 
plexity  theoretic  properties  of  automatic  structures  are  amendable  to  systematic 
investigation.  Here  are  a  few  open  questions. 

Question  1.  Characterize  the  first  countable  ordinal  which  does  not  have 
any  automatic  presentation. 

In  section  2,  we  gave  automatic  presentations  for  ordinals 

Question  2.  Give  an  algebraic  or  model- theoretic  characterization  of  au¬ 
tomata  presentable  structures. 

In  section  5  we  characterized  strongly  automata  representable  structures  in 
terms  of  complete  relations. 

Question  3.  Characterize  decidable  first  order  theories  for  which  every 
countable  model  has  an  automatic  reprsnetation. 

Section  4  shows  that  the  first  order  theory  of  every  automatic  structure  is 
decidable.  On  the  other  hand  it  is  known  that  every  decidable  first  order  theory 
possesses  a  decidable  structure  [7].  It  can  also  be  proved  that  finding  the  truth 
value  of  a  fully  quantified  automaton  recognizable  predicate  is  exponential  in 
the  size  of  the  automaton  recognizing  the  relation  [6]. 

Question  4.  Characterize  automatic  isomorphism  types  of  automatic  struc¬ 
tures  over  a  fixed  domain. 

In  the  last  section,  we  proved  that  if  we  do  not  fix  domains,  then  any  struc¬ 
ture  with  exactly  one  isomorphism  type  is  finite.  We  do  not  know,  however, 
what  effect  a  fixed  domain  has  on  automatic  isomorphism  types. 

In  this  paper  we  dis  not  consider  presentations  of  structures  using  tree  au¬ 
tomata.  One  can  develop  and  study  tree  automata  presentatable  structures.  Of 
course,  an  approach  based  on  tree  automata  would  cover  this  paper  and  possess 
the  same  positive  features  of  decidability.  However  we  decided  to  present  clear 
definitions  and  examples  based  on  the  simple  computational  structure  of  finite 
automata.  The  investigation  of  tree  automata  presentable  structures  we  have 
under  development.  [20]. 
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Abstract.  We  introduce  a  restricted  version  of  second  order  logic  SO*^ 
in  which  the  second  order  quantifiers  range  over  relations  that  are  closed 
under  the  equivalence  relation  =*  of  k  variable  equivalence,  for  some  k . 
This  restricted  second  order  logic  is  an  effective  fragment  of  the  infinitary 
logic  j  which  differs  from  other  such  fragments  in  that  it  is  not  based 
on  a  fixpoint  logic.  We  explore  the  relationship  of  SO*^  with  fixpoint 
logics,  showing  that  its  inclusion  relations  with  these  logics  are  equivalent 
to  problems  in  complexity  theory.  We  also  look  at  the  expressibility  of 
NP-complete  problems  in  this  logic. 


1  Introduction 

In  recent  years,  much  research  in  finite  model  theory  has  focused  on  its  con¬ 
nections  with  computational  complexity  theory.  It  turns  out  that  there  is  a 
close  relationship  between  the  computational  complexity  of  a  problem,  i.e.  the 
amount  of  resources  needed  to  solve  the  problem  on  some  machine  model  of 
computation,  and  its  descriptive  complexity,  i.e.  the  kinds  of  ^‘logical  resources” 
that  are  needed  to  describe  the  problem.  The  paradigmatic  result  establishing 
a  connection  between  descriptive  and  computational  complexity  is  the  result  of 
Fagin  [11]  which  shows  that  the  properties  of  finite  structures  that  are  defin¬ 
able  by  sentences  of  existential  second  order  logic  are  exactly  those  that  are  in 
the  complexity  class  NP.  This  was  extended  by  Stockmeyer  [23]  to  a  tight  corre¬ 
spondence  betwen  second  order  logic  and  the  polynomial  time  hierarchy.  Further 
work  along  these  lines  has  established  logical  characterizations  for  a  wide  range 
of  complexity  classes  (see,  for  instance,  [16]). 

However,  some  of  the  results  equating  logical  expressibility  to  computational 
complexity  require  the  finite  structures  to  have  a  built-in  linear  order.  That  is, 
the  exact  correspondence  between  expressibility  in  a  logic  and  solvability  within 
given  resource  bounds  does  not  hold  over  the  class  of  all  finite  structures,  but  is 
restricted  to  those  structures  which  have  a  linear  order  as  one  of  their  relations. 
Thus,  for  instance,  Immerman  [15]  and  Vardi  [24]  independently  showed  that  the 
extension  FP  of  first  order  logic  by  means  of  a  fixpoint  operator  characterizes  the 
class  PTIME  on  the  class  of  ordered  structures.  No  such  logical  charactrerization 
of  PTIME  is  known  for  arbitrary  finite  structures.  Similarly,  by  results  of  Vardi 
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[24]  and  Abiteboul  and  Vianu  [2]  it  is  known  that  the  extension  PFP  of  first 
order  logic  by  a  partial  fixpoint  operator  characterizes  the  class  PSPACE  on  the 
class  of  ordered  structures. 

In  general,  FP  is  strictly  weaker  than  PTIME.  That  is  to  say,  while  every 
property  expressible  in  FP  is  decidable  in  PTIME,  there  are  PTIME  properties 
that  are  not  expressible  in  FP.  The  same  holds  true  of  PFP  and  PSPACE. 
Nevertheless,  Abiteboul  and  Vianu  [3]  were  able  to  show  that  FP  =  PFP,  if 
and  only  if,  PTIME  =  PSPACE.  Thus,  even  though  we  do  not  have  a  logical 
characterization  of  the  class  PTIME  over  all  finite  structures,  the  open  com¬ 
plexity  theoretic  question  about  the  separation  of  PTIME  and  PSPACE  can  be 
translated  to  an  equivalent  question  about  the  expressive  power  of  two  logics  on 
the  class  of  all  finite  structures.  Extending  this  work,  Abiteboul  ei  al  [1]  defined 
a  variety  of  fixpoint  logics  and  showed  that  for  a  range  of  complexity  classes 
between  PTIME  and  EXPTIME,  open  questions  about  the  separations  of  these 
classes  are  equivalent  to  separations  of  corresponding  fixpoint  logics.  They  also 
gave  characterizations  of  these  fixpoint  logics  in  terms  of  computability  on  a 
relational  machine  model  of  computation,  establishing  a  general  result  showing 
that  inclusion  relations  among  relational  complexity  classes  mirror  those  among 
the  usual  computational  complexity  classes. 

The  interest  in  fixpoint  logics  has  also  focused  attention  on  the  infinitary  logic 
with  finitely  many  variables  -  .  All  of  the  fixpoint  logics  mentioned  above 

can  be  seen  as  fragments  of  .  Recently,  considerable  effort  has  been  devoted 
to  understanding  the  model  theory  of  on  finite  structures  (see,  for  instance 
[10,  17,  18]).  One  of  the  reasons  for  this  is  that  definability  in  has  an  elegant 
characterization  in  terms  of  two-player  pebble  games.  Indeed,  this  has  been  the 
main  tool  used  so  far  in  establishing  inexpressibility  results  for  fixpoint  logic. 
The  logic  has  also  proved  a  vehicle  for  introducing,  important  notions  from 
classical  model  theory  such  as  elementary  equivalence  and  element  types  in  to 
finite  model  theory  in  a  meaningful  way,  by  restricting  the  number  of  variables. 
A  systematic  study  of  the  I: -variable  elementary  equivalance  relation  =*  was 
,  undertaken  in  [10].  It  is  felt  that  the  translation  of  important  open  questions  in 
complexity  theory  into  questions  about  fragments  of  ,  as  in  [1]  for  instance, 
provides  a  greater  opportunity  for  the  application  of  model-theoretic  techniques 
to  these  questions. 

In  this  paper,  we  continue  the  study  of  the  model  theory  of  by  defining 
a  restricted  version  of  second-order  logic  SO"  that  is  contained  within  . 
This  is  obtained  by  restricting  the  interpretation  of  second  order  quantifiers  to 
relations  closed  under  the  equivalence  relation  ,  for  some  k .  We  show  that 
the  existential  fragment  of  this  logic  is  the  class  relational  NP,  while  SO"  itself 
coincides  with  relational  PH.  This  establishes  results  in  the  style  of  [3]  for  all 
levels  of  the  polynomial  time  hierarchy.  Moreover,  these  are  of  a  somewhat  differ¬ 
ent  character  to  the  results  in  [1]  in  that  the  chatacterizations  are  not  in  terms 
of  fixpoint  logics.  We  also  discuss  the  expressibility  of  NP-complete  problems 
in  our  restricted  second  order  logic,  giving  examples  of  natural  problems  that 
can  be  expressed  in  this  way,  as  well  as  illustrating  techniques  for  establishing 
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lower  bounds  by  showing,  for  instance,  that  3-colour  ability  cannot  be  expressed 

in  . 


2  Background  and  Notation 

In  this  section,  we  fix  our  notation  and  examine  the  necessary  background  ma¬ 
terial.  We  assume  familiarity  with  the  basic  notions  of  predicate  logic,  as  well  as 
basic  definitions  from  complexity  theory. 

A  signature  cr  is  a  finite  sequence  of  relation  symbols  (iii, . .  • , -Rs),  with 
associated  arities  ai , . . . ,  o, .  A  (t -structure  21  =  (A,  Rf , . . . ,  R^)  consists  of  a 
finite  set  A ,  referred  to  as  the  universe  or  domain  of  21  and  interpretations  of 
the  relation  symbols  in  cr  as  relations  on  A,  i.e.,  ilp  C  A*** . 

An  m-ary  query  g  is  a  map  that  takes  structures  over  some  fixed  signature 
or  to  m-ary  relations  on  the  domains  of  the  structures  and  is  closed  under  iso¬ 
morphisms.  That  is,  for  any  m-tuple  t  in  a  structure  2t  and  any  isomorphism 
/  from  21  to  !B,  t  €  g(2t)  if,  and  only  if,  /(<)  €  g(<B) .  For  instance,  a  first  order 
formula  with  m  free  variables  defines  an  m-ary  query.  A  0-ary  query,  also  called 
a  Boolean  query,  is  a  map  from  the  class  of  cr -structures  to  the  set  {True,  False} , 
and  can  be  identified  with  an  isomorphism  closed  class  of  or -structures.  In  gen¬ 
eral,  we  say  that  a  query  is  expressible  in  a  logic  L  if  there  is  some  formula  of 
L  that  defines  it.  By  abuse  of  notation,  we  will  also  use  L  to  denote  the  class  of 
queries  that  are  definable  in  the  logic  L .  When  we  speak  of  the  computational 
complexity  of  a  query  q ,  we  mean  the  complexity  of  deciding,  given  a  structure 
2t  and  a  tuple  t  from  the  domain  of  21,  whether  it  is  the  case  that  t  E  g(2l). 
The  measure  of  the  size  of  the  input  is  the  cardinality  of  the  domain  of  2t. 

We  say  that  a  logic  L  captures  a  complexity  class  C  if  every  query  that 
is  expressible  by  a  formula  of  L  is  in  the  complexity  class  C  and  conversely, 
every  query  that  is  in  C  is  expressible  by  a  formula  of  L .  We  also  say  that  L 
captures  (7  on  a  class  of  structures  S  when  the  equivalence  between  L  and  C 
holds  for  queries  whose  domain  is  restricted  to  5.  Note  that  this  usage  of  .the 
term  “capture”  is  not  the  same  as  in  [1]. 

We  write  17}  for  the  collection  of  second  order  sentences  in  prenex  normal 
form,  in  which  all  second  order  quantifiers  precede  the  first  order  quantifiers,  and 
which  contain  only  existential  second  order  quantifiers.  Fagin  [11]  proved  that  17} 
captures  NP.  This  result  was  extended  by  Stockmeyer  [23]  to  show  that  second 
order  logic  captures  the  polynomial  time  hierarchy.  Indeed,  the  correspondence 
between  second  order  logic  and  the  polynomial  time  hierarchy  holds  level  by 
level.  That  is,  if  17^^!  denotes  the  collection  of  sentences  of  second  order  logic 
containing  n  alternations  of  second  order  quantifiers  starting  with  an  existential 
quantifier,  then  17^  captures  IJP  ,  the  nth  level  of  the  polynomial  time  hierarchy. 

2.1  Fixpoint  Logics 

Let  <p  he  a.  formula  with  free  individual  variables  among  xi,...,ar,n,  in  the 
signature  a  extended  with  an  additional  m-ary  predicate  symbol  R.  On  o*- 
structures,  defines  an  operator  mapping  m-ary  relations  to  m-ary  relations. 
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Thus,  given  a  <t -structure  Ql  and  an  m-ary  relation  P  in  21,  we  define 
to  be  {5  I  (21,  P)  y?[s]}.  If  this  operator  is  monotone,  that  is,  for  every  P 
and  Q  such  that  P  C  Q,  C  then  it  has  a  least  fixed  point.  While 

monotonicity  is  a  semantic  property,  there  is  a  syntactic  condition  on  (p  that 
guarantees  that  the  corresponding  operator  is  monotone.  Namely,  if  <p  is  R- 
positive,  that  is  all  occurrences  of  P  in  are  in  the  scope  of  an  even  number 
of  negations,  then  the  operator  defined  by  (p  is  monotone.  We  write  LFP  for 
the  closure  of  first  order  logic  under  the  operation  of  taking  least  fixed  points 
of  positive  formulas.  Immerman  [15]  and  Vardi  [24]  independently  showed  that 
LFP  captures  the  complexity  class  PTIME  over  the  class  of  structures  which 
include  a  linear  order  as  one  of  their  relations. 

If  p  defines  a  monotone  operator,  then  its  least  fixed  point  in  a  structure  21 
can  be  obtained  by  iterating  the  operator  as  follows.  Define  to  be  the  empty 
relation  0,  and  define  to  be  .  Because  the  operator  is  monotone, 

this  sequence  of  relations  is  increasing,  and  if  21  has  cardinality  n ,  then  for  some 
i  <n^  ,  p*  =  9?*+^ .  This  p*  is  then  the  least  fixed  point  of  v?.  A  similar  iteration 
can  be  defined  even  when  p  does  not  define  a  monotone  operator  by  taking  at 
each  stage  the  union  with  the  previous  stage.  That  is,  define  . 

The  resulting  sequence  of  relations  is  increasing  for  any  p ,  and  once  again  reaches 
a  fixed  point  for  some  i  <  .  This  is  the  inflationary  fixed  point  of  p.  IFP 

is  defined  to  be  the  closure  of  first  order  logic  under  the  operation  of  taking 
inflationary  fixed  points  of  arbitrary  formulas.  Clearly,  for  positive  formulas,  the 
least  fixed  point  and  the  inflationary  fixed  point  coincide.  Moreover,  Gurevich 
and  Shelah  [13]  showed  that  for  every  formula  p^  the  inflationary  fixed  point  of 
p  is  definable  by  a  formula  of  LFP.  It  follows  that  the  two  logics  IFP  and  LFP 
are  equivalent  on  finite  structures.  For  the  rest  of  this  paper,  we  will  use  the 
notation  FP  to  denote  the  logic  IFP. 

Immerman  [15]  established  a  normal  form  for  formulas  of  LFP.  This,  along 
with  the  result  of  Gurevich  and  Shelah  mentioned  above  provides  similar  normal 
forms  for  IFP  (see  also  [2]): 

Theorem  1,  For  every  formula  p  0/  FP  there  is  a  formula  which  is  the 
inflationary  fixed  point  of  a  first  order  formula,  such  that  p  is  equivalent  to 
35^  and 

Indeed,  we  can  even  require  that  the  first  order  formula  of  which  ^  is  the  fixpoint 
is  itself  existential  (see  [2]). 

Consider  now  an  arbitrary  formula  p  that  does  not  necessarily  define  a 
monotone  operator.  The  sequence  of  stages  defined  by  taking  =  0  and 
is  not  necessarily  increasing  and  may  or  may  not  converge  to 
a  fixed  point.  However,  if  there  is  an  i  such  that  ,  then  there  is  such 

an  f  <  2”  .  The  partial  fixed  point  of  p  is  defined  to  be  p^  for  i  such  that 
p*  —  ,  if  such  an  i  exists,  and  empty  otherwise.  PFP  denotes  the  closure  of 

first  order  logic  under  an  operation  defining  the  partial  fixed  point  of  formulas. 
Abiteboul  and  Vianu  [2]  showed  that  PFP  is  equivalent  to  the  language  while 
introduced  by  Chandra  and  Harel  [6].  Vardi  [24]  showed  that  the  language  while 
captures  the  class  PSPACE  on  the  collection  of  structures  with  a  linear  order. 
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There  is  an  apparently  more  general  form  of  inductive  definition,  where 
a  query  is  defined  by  simultaneous  induction  of  a  number  of  formulas.  Let 
So,...ySi  be  a  sequence  of  relation  symbols  that  do  not  occur  in  the  signa¬ 
ture  or,  with  associated  arities  oo, •  Further,  let  y>o»  *  •  •  >  be  a  sequence 
of  formulas  in  the  signature  formed  by  extending  <t  by  where  (pj 

defines  a  query  of  arity  aj.We  define  the  stages  of  the  simultaneous  induction 
of  the  sequence  •  •  • » V?/)  by: 

¥>“  =  0 

. ipj  U  for  the  inflationary  case). 

The  sequence  reaches  a  fixed  point  if  there  is  an  i  such  that  (pj  =  (pj^^  for  all 
0  <  j  <  / ,  and  the  relation  defined  by  the  fixpoint  is  then  (pQ . 

Moschovakis  [21]  showed  that  allowing  simultaneous  inductions  does  not  in¬ 
crease  the  expressive  power  of  fixpoint  logics  (see  also  [19]  for  a  discussion  and 
[2]  for  the  case  of  partial  fixpoints): 

Theorem  2.  Every  query  defined  by  a  simultaneous  inflationary  (resp.  partial) 
induction  is  definable  in  FP  (resp.  PFP 

In  the  light  of  Theorem  2,  we  will  use  simultaneous  inductions  in  this  paper 
wherever  it  makes  the  exposition  clearer. 

Abiteboul  et  al.  [1]  extended  the  above  framework  of  fixpoint  logics  by  defin¬ 
ing  a  range  of  fixpoint  logics  obtained  by  varying  two  parameters  -  the  control 
mechanism  and  the  semantics  of  the  fixpoint  iteration.  The  control  mechanism 
can  be  deterministic,  non-deterministic  or  alternating,  and  the  semantics  can  be 
inflationary  or  non-inflationary.  The  two  fixpoint  logics  considered  above,  FP 
and  PFP,  in  this  terminology,  are  both  deterministic  in  their  control,  with  infla¬ 
tionary  and  non-inflationary  semantics,  respectively.  We  will  now  consider  the 
non-deterministic  inflationary  fixpoint  logic,  introduced  by  Abiteboul  et  al  We 
denote  this  logic  NFP,  for  non-deterministic  fixpoint  logic.^ 

Given  two  formulas  stud  in  a  signature  tr  extended  by  an  additional 
m-ary  predicate  we  define  in  any  cr -structure  21,  a  sequence  of  stages  of  the 
pair  (v?o,V^i)  indexed  by  binary  strings: 

=0,  for  the  empty  string  e 

<p‘  °=  ifi‘  u 

We  now  define  the  non-deteministic  fixed  point  of  the  pair  (^o,  ^i)  in  the  struc¬ 
ture  21  to  be  U«e{o,i}*  •  "^be  logic  NFP  is  the  closure  of  first  order  logic  under 

the  operation  of  taking  non-deterministic  fixed  points,  with  the  proviso  that  the 
fixpoint  operator  does  not  occur  within  the  scope  of  a  negation. 

We  observe,  without  proof,  that  Theorems  1  and  2  extend  directly  to  NFP  as 
well.  That  is,  we  define  a  simultaneous  non-deterministic  induction  by  a  sequence 

This  notation  is  different  from  [1],  where  NFP  denotes  the  logic  we  call  PFP. 
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of  pairs  of  formulas:  tpi  =  0  *<  i  <  /  in  a  signature  <t  extended  with 

new  relation  symbols  The  stages  of  this  induction  on  a  or -structure 

2t  are  defined  as  follows: 


<pI  =0,  for  the  empty  string  c 

y>?  i=  (p’  u  ' 


The  non-deteministic  fixed  point  of  the  sequence  is  given  by  U.6{o,i}-  V’o-  The 
following  lemma  is  proved  in  the  same  way  as  Theorem  2. 


Lemma  3,  Every  query  defined  by  a  simultaneous  non- deterministic  induction 
is  definable  in  NFP. 


Similarly,  the  proof  of  the  normal  form  result,  Theorem  1,  can  also  be  extended 
tp  NFR 

Lemma  4  •  Every  formula  o/ NFP  is  equivalent  to  a  formula  of  the  form  3x(p, 
where  (p  is  the  non- deterministic  fixed  point  of  a  pair  of  first  order  formulas. 


2.2  Infinitary  Logic 

The  infinitary  logic  Looa»  is  obtained  by  closing  first  order  logic  under  conjunc¬ 
tions  and  disjunctions  of  arbitrary  (not  just  finite)  sets  of  formulas.  This  logic 
is  of  little  use  in  the  study  of  finite  models,  since  every  query  on  the  class  of 
finite  structures  is  expressible  in  Loow  •  However,  the  restriction  of  Loolj  where 
we  only  allow  finitely  many  variables  to  appear  in  any  given  formula,  has  proved 
to  be  of  great  value  in  studying  the  expressive  power  of  fixed  point  logics  on 
finite  structures. 

More  formally,  let  denote  the  class  of  formulas  of  Loou;  in  which  all 

variables  (free  or  bound)  are  among  Also,  let 

The  logic  was  introduced  by  Barwise  [4]  in  order  to  study  inductive  deftni- 
tions  on  infinite  structures.  It  was  shown  by  Rubin  [22]  that  for  a  fixed  infinite 
structure,  the  least  fixed  point  of  any  first  order  operator  is  expressible  in  . 
A  similar  result  was  obtained  for  the  class  of  all  finite  structures  by  Kolaitis 
and  Vardi  [18],  who  showed  that  in  this  case,  both  FP  and  PFP  can  be  seen  as 
fragments  of  .  This  also  applies  to  NFP,  giving  us  the  following  picture: 

FP  C  NFP  C  PFP  C 

The  last  containment  in  the  above  is  a  proper  one,  since  Kolaitis  and  Vardi  [18] 
show  that  there  are  noh-recursive  queries  that  can  be  expressed  in  ,  while 
every  query  definable  in  PFP  is  computable  in  PSPACE.  Indeed  one  can  show 
that  just  as  Loow  is  complete  in  its  expressive  power,  so  is  complete  on 
ordered  structures  (for  a  fuller  discussion  of  this,  see  [8]): 

Propositions.  For  every  signature  <t,  there  is  a  k  ^  oj  such  that  every  query 
on  ordered  <T  -structures  is  expressible  in  . 
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We  write  for  the  first  order  fragment  of  ,  i.e.  the  formulas. of  first  order 
logic  that  contain  only  the  variables  xi, . . . ,  xjb . 

Recall  that  for  a  structure  and  a  tuple  s  of  elements  of  21 ,  the  first  order 
type  of  s  in  21,  denoted  Type(2l,  s)  is  the  set  of  formulas  ^  such  that  21 1=  y>[s] . 
The  following  variant  of  this  notion  was  introduced  in  [10],  and  has  proved  to 
be  very  useful  in  studying  expressibility  in  . 

Definition  6.  For  any  structure  21  and  a  tuple  s  of  elements  of  21,  Type^(2l,  s) 
denotes  the  set  of  formulas  9?  of  L*  such  that  21 1=  y?[s] . 

We  write  (2t,s)  ^  (25,  t)  to  denote  Type* (21, 5)  =  Type*(25,<).  We  also  write 
jk-size(2t)  to  denote  the  number  of  equivalence  classes  of  the  relation  =*  in  the 
structure  2t. 

Kolaitis  and  Vardi  [18]  showed  that  the  equivalence  relation  =*  coincides, 
on  finite  structures,  with  the  apparently  stronger  notion  of  equivalence  in  . 
That  is,  they  showed  that  if  21  and  25  are  finite  structures  and  (21,  s)  =*  (®,<) 
then  for  every  formula  (p  G  ,  2t  ^  ^[s]  if  and  only  if  |=  (p[t].  Kolaitis 
and  Vardi  [18]  also  showed  that  a  query  q  is  definable  in  if,  and  only 

if,  it  is  closed  under  the  relation  =*,  i.e.  if  s  G  g(2l)  and  (21,  s)  =*  (25,  t) 
then  t  G  g(25) .  It  was  shown  in  [10]  that  if  21  is  a  finite  structure,  there  is  a 
formula  (p  G  Type* (21,  s)  such  that  for  any  structure  25,  |=  (p[t]  if  and  only 

if  (2t,s)=*(©,t). 

The  equivalence  relation  =*  has  an  elegant  characterization  in  terms  of 
Ehrenfeucht-Frai’sse  style  pebble  games,  essentially  given  by  Barwise  [4]  (see 
also  [14]).  The  game  board  consists  of  two  structures  21  and  25  and  a  supply  of 
k  pairs  of  pebbles  (a<,  6,*),  1  <  z  <  A:.  The  pebbles  ai, . . . ,  a/  are  initially  placed 
on  the  elements  of  an  /-tuple  s  in  21,  and  the  pebbles  61, . . . ,  6^  on  a  tuple  t  in 
!8.  There  are  two  players,  Spoiler  and  Duplicator.  At  each  move  of  the  game, 
Spoiler  picks  up  a  pebble  (either  an  unused  pebble  or  one  that  is  already  on  the 
board)  and  places  it  on  an  element  of  the  corresponding  structure.  For  instance 
he  might  take  pebble  6,-  and  place  it  on  an  element  of  25,  Duplicator  must  re¬ 
spond  by  placing  the  other  pebble  of  the  pair  in  the  other  structure.  In  the  above 
example,  she  must  place  a,-  on  an  element  of  21.  If  at  the  end  of  the  move  the 
partial  map  /  :  21  — >  25  given  by  a,*  hi  is  not  a  partial  isomorphism,  then 
Spoiler  has  won  the  game,  otherwise  it  can  continue  for  another  move.  Duplica¬ 
tor  has  a  strategy  to  avoid  losing  for  n  moves,  starting  with  the  initial  position 
(21,  s)  and  (25,  t)  if,  and  only  if,  (21,  s)  and  (!B,t)  cannot  be  distinguished  by  any 
formula  of  L*  of  quantifier  rank  n  or  less.  Hence,  if  Duplicator  has  a  strategy 
to  play  the  game  indefinitely  without  losing,  then  (21,  s)  =*  (25,  t). 

The  relation  =*  is  itself  uniformly  definable  in  FP  [10,  17]: 

Theorem  7.  There  is  a  formula  1]  0/  FP,  with  2k  free  variables^  such  that  on 
any  finite  structure  21,  given  two  k -tuples  s  and  t  in  21,  21  [=  z?[5,  t]  if,  and 
only  if,  (a,s)  =*’  (Ql.t). 

Moreover,  we  can  also  write  a  formula  A  of  FP  which  uniformly  orders  =* 
equivalence  classes  (see  [3,  10]).  That  is,  on  any  finite  structure  21,  A  defines  a 
pre-order  such  that  the  corresponding  equivalence  relation  is  =* : 
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Theorems,  There  is  a  formula  A  of  FT,  with  2k  free  variables,  suck  that  on 
any  finite  structure  A  defines  a  reflexive  and  transitive  relation  on  k- 
tuples  such  that  for  every  two  k -tuples  s  and  t,  either  s  t  or  t  s  and, 
both  s  t  and  t  s  hold  if  and  only  if  (21,  s)  =*  (Qt,  <) . 

Thus,  A  can  be  seen  as  defining  a  total  order  on  the  equivalence  classes  of  . 
The  FP  definition  of  this  order  allows  us  to  define  an  FP  reduction  which  maps 
any  structure  21  to  a  quotient  structure  21/=^  which  is  linearly  ordered.  Using 
such  a  reduction,  Abiteboul  and  Vianu  [3]  showed  that  FP  =  PFP  if  and  only 
if  PTIME  =  PSPACE  (see  also  the  exposition  in  [10]). 

Abiteboul  et  al.  [1]  extend  this  by  showing  that  the  logic  NFP  captures  the 
relational  complexity  class  NPr ,  whereby  it  follows  that: 

FP  =  NFP  if  and  only  if  PTIME  =  NP;  and 

NFP  =  PFP  if  and  only  if  NP  =  PSPACE. 

3  A  Restricted  Second  Order  Logic 

The  fixpoint  logics  can  be  viewed  as  effective  fragments  of  ,  as  we  saw  in 
Section  2.2.  In  this  section,  we  explore  a  different  way  of  obtaining  an  effective 
fragment  of  ,  by  a  restricted  form  of  second  order  quantification.  This  pro¬ 
vides  a  logical  characterization  of  some  relational  complexity  classes  that  is  not 
based  on  a  fixpoint  logic,  and  is  closer  in  spirit  to  Fagin’s  characterization  of  NP. 

Definition  9. 

~  For  an  /-ary  relation  symbol  R,  and  k  >  /,  we  define  the  second  order 
quantifier  3^R  to  have  the  following  semantics:  %  \=.  3^ R(p  if  there  is  an 
X  C  such  that  X  is  closed  under  the  equivalence  relation  in  21 ,  and 
{%X)  (=  As  usual,  abbreviates 

-  denotes  the  class  of  formulas  of  the  form  3^^Ri  . .  .3^”^Rm(p,  where  (p 
is  first  order. 

-  denotes  the  class  of  formulas  of  the  form  . .  ,y^”^Rm<Pi  where  ip 

is  first  order. 

-  denotes  the  class  of  formulas  of  the  form  3^^Ri . .  ,3^^Rm^f  where  p 
ism.-. 

~  denotes  the  class  of  formulas  of  the  form  where 

ypisri--. 

The  logic  is  a  restricted  version  of  second  order  logic  which  forms  an 
effective  fragment  of  .  In  Theorem  12  below,  we  will  see  that  it  is  in  fact 
contained  in  PFP.  We  begin  by  establishing  its  relationship  with  the  usual 
second  order  logic. 

Lemma  10,  For  every  n,  C  and  Q 
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Proof.  We  present  the  proof  for  the  i7-classes.  The  proof  for  the  i7 -classes  is 
analogous. 

Let  (p  £  be  a  formula  3^^Ri . .  .Q^^iRqxj^ ,  where  Q  is  3  or  V,  depending 
on  whether  n  is  odd  or  even.  Clearly,  the  query  defined  by  <p  can  be  expressed 
as: 

/\  yHm,  (1) 

l<i<j 

where  asserts  that  Ri  is  =** -closed.  By  Theorem  7  each  7*  is  definable 

in  FP .  Since  FP  C  fl  ,  it  follows  that  the  query  (1)  is  definable  in  . 

Theorem  11.  On  ordered  structures,  for  every  n  £lo,  and  = 

n\. 

Proof  Since  Lemma  10  holds  on  arbitrary  structures,  it  holds  on  ordered  struc¬ 
tures,  in  particular.  Thus,  we  only  need  to  show  the  inclusions  E}^  C  and 

It  follows  from  Proposition  5  that  for  every  signature  <7  there  is  a  such 
that,  if  21  is  an  ordered  cr -structure  and  R  is  any  /-ary  relation  on  21  for  /  <  ^<7 , 
then  R  is  -closed.  Let  p  £  E\  be  a  <r -sentence  3Ri . .  .QRqxj).  For  each  Ri 
of  arity  a*- ,  let  ki  =  max(ai-,  ^<7).  Then  it  is  easily  seen  that  (p  is  equivalent,  on 
ordered  structures,  to  the  sentence 

3^^Ri...Q^'^Rq'il). 

The  proof  for  i7^  sentences  is  similar. 

Theorem  11  establishes  that  the  restricted  second  order  logic  is  not 
really  restricted  on  ordered  structures.  In  what  follows,  we  establish  the  rela¬ 
tionship  of  SO**'  to  the  fixpoint  logics. 

Theorem  12.  SO**'  C  PFP. 

Proof.  It  suffices  to  show  that,  given  a  formula  ip  of  PFP ,  there  is  a  formula  p 

of  PFP  equivalent  to  3* iiV’- 

Any  relation  P  that  is  -closed  on  a  structure  21  can  be  seen  as  a  set  of 
equivalence  classes.  Thus,  the  pre-order  <*  of  Theorem  8,  being  a  linear  order 
on  the  collection  of  equivalence  classes,  induces  a  lexicographical  ordering  of 
all  s*' -closed  relations  on  2t.  Moreover,  using  the  FP  formula  defining  <*  ,  we 
can  write  an  FP  formula  ^'(P)  which  defines,  for  any  -closed  relation  P  of 
arity  /,  the  lexicographically  next  such  relation. 

We  assume  that  the  formula  3^  Rip  has  free  individual  variables  among 
xi, . .  and  therefore  defines  an  m-ary  query.  We  define  this  query  by  means 
of  a  siihultaneous  induction  of  two  formulas.  We  therefore  have  two  induction 
variables  S  and  R  of  arity  m  and  /,  repectively.  At  succesive  stages  of  the  in¬ 
duction  R  takes  on,  in  lexicographical  order,  the  values  of  =* -closed  relations, 
reaching  a  fixed  point  when  it  contains  all  /-tuples.  At  the  same  time  S  accu¬ 
mulates  the  m-tuples  that  satisfy  ip{R) .  Formally,  we  define  the  formulas  ps 
and  pR  as  follows: 
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(ps  =  S{x)  V  ^{R) 

(pR  =  Vyi2(j/)  V  {3y->R{y)  A  i/(i2)). 

It  can  be  verified  that  the  simultaneous  partial  fixed  point  of  this  induction 
yields  the  query  3^ Rip .  Therefore,  by  Theorem  2,  we  have  a  formula  of  PFP 
that  expresses  this  query. 

We  now  show  that  the  existential  fragment  of  SO^  is,  in  fact,  equivalent  to 
NFP .  In  the  next  two  lemmas,  we  state  the  crucial  results  for  proving  the  two 
directions  of  the  equivalence  of  and  NFP. 

Lemma  13,  For  any  pair  of  first  order  formulas  po  and  pi ,  there  is  a  formula 
of  that  defines  the  non- deterministic  fixed  point  of  (potpi). 

Proof  We  note  first  that  by  the  inflationary  nature  of  the  non-deterministic 
fixpoint  operator,  if  si  is  a  prefix  of  S2 ,  then  C  in  any  structure  Qt .  Thus, 
if  we  consider  any  increasing  sequence  of  binary  strings,  then  the  corresponding 
sequence  of  stages  is  increasing.  It  follows  that,  if  we  let  k  be  the  maximum  of 
the  number  of  distinct  variables  in  po  and  pi ,  then  for  binary  strings  s  such 
that  length(s)  >  fc-size(2t),  p‘  =  p^‘^  =  p^‘^ .  Consider  the  formula: 

3^^03^-^^Rip 

with  free  individual  variables  xi, . . . ,  arm »  where  ip  asserts  that: 

-  O  is  a  pre-order  of  ^-tuples;  . 

“  72°  =  0  and  for  every  z,  either  R*"^^  —  R*\Jpq*  or  R*"^^  =  R*Upi'* ,  where 
R'  is: 

{t  |i2(s,  i)  for  some  s  in  the  zth  equivalence  class  determined 
by  the  pre-order  O}; 

and 

-  R^{xi , . . . ,  Xm) ,  where  m  is  the  length  of  the  pre-order  O . 

It  can  be  verified  that  this  formula  expresses  the  non-deterministic  fixpoint  of 

the  pair 

In  the  other  direction,  we  have  the  following  lemma. 

Lemma  14>  If  ip  is  a  formula  of  NFP,  then  there  is  a  formula  of  NFP  that  is 
equivalent  to  3^ Rip . 

Proof  For  simplicity,  we  assume  that  the  arity  of  7^  is  A:  and  that  the  free 
individual  variables  in  ip  are  among  xi, . . . ,  Xm . 

As  in  the  proof  of  Theorem  12,  we  are  going  to  use  the  FP  definition  of  the 
order  <*  of  =*  equivalence  classes.  Intuitively,  we  want  to  define  an  induc¬ 
tion  that  steps  along  this  order  and  at  each  stage  decides  non-deterministically 
whether  or  not  to  include  the  current  equivalence  class  in  the  relation  72.  For 
this,  we  essentially  need  to  maintain  three  relations:  one,  5,  to  count  the  equiv¬ 
alence  classes  that-  have  been  visited,  one  to  include  those  equivalence  classes 
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that  have  been  chosen  to  be  in  and  finally  one,  P,  to  construct  the  relation 
defined  by  ip ,  given  the  candidate  R,  We  do  this  by  a  simultaneous  induction  of 
three  pairs  of  formulas,  {(pp,o,(pp,i)y{(pR,o,(pR^i)  and  (v?5,o,  with  relation 
symbols  R  and  S  of  arity  k  and  P  of  arity  m.  Note  that  in  the  definition 
below,  (pp^o  =  <pp^i  and  (ps,o  =  <ps,i  so  the  non-determinism  is  confined  to  the 
pair  {(pR,o,fpR,i)^ 

V?p,o  =  V?p,i  =  Vy5(y)  A  ip{R) 

<Ps,o  s  (ps,i  =  Vj/(A(y,  x)  S{y)) 

(PR,0  =  X  ^  JT 

In  the  above  A  is  the  FP  formula  in  Theorem  8  that  defines  the  pre-order  . 
It  is  clear  that  any  inflationary  fixpoint  can  be  expressed  as  a  non-deterministic 
fixpoint  (simply  by  taking  =  V^i)-  A  slight  complication  arises  because  in 
the. above  formulas  A  appears  within  the  scope  of  a  negation  symbol.  However, 
by  Theorem  1,  we  know  that  the  negation  of  an  FP  formula  can  be  expressed 
without  the  fixpoint  operator  appearing  inside  the  scope  of  a  negation. 

It  can  be  verified  that  the  simultaneous  non-deterministic  fixpoint  of  the 
above  system  defines  the  query  3^ Rip .  Therefore,  by  Lemma  3,  there  is  a  formula 
of  NFP  that  defines  the  same  query. 

The  following  theorem  is  immediate  from  Lemmas  4,  13,  and  14. 

Theorem  1 5.  =m?. 

Remark.  The  proof  of  Theorem  15  can  be  extended  to  show  that,  if  we  close 
the  logic  NFP  simultaneously  under  negation  and  the  operation  of  taking  non- 
deterministic  fixpoints,  we  obtain  a  logic  equivalent  to  S0‘*' .  Moreover,  the  alter¬ 
nations  of  negations  and  fixpoints  correspond  exactly  to  the  second  order  quanti¬ 
fier  alternations  in  .  Similarly,  Abiteboul  et  al.  [1]  also  define  an  alternating 
inflationary  fixpoint  logic,  which  they  show  to  be  equivalent  to  PFP.  One  can 
show  that  the  fragment  of  this  logic  obtained  by  allowing  only  a  bounded  num¬ 
ber  of  alternations  is  equivalent  to  .  Once  again,  the  number  of  alternations 
corresponds  exactly  to  the  number  of  alternations  of  second  order  quantifiers  in 
SO^". 

The  following  corollaries  follow  immediately  from  Theorem  15. 

Corollary  16.  FP  C  tJ’*"  n  . 

Corollary  17.  FP  =  if  and  only  if  PTIME  =  NP . 

Corollary  18.  rj’"  =  PFP  if  and  only  if  NP  =  PSPACE. 

Moreover,  an  application  of  the  same  methods  also  gives  us: 

Theorem  19.  SO^*'  =  PFP  if  and  only  if  PH  =  PSPACE. 
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By  Corollaries  17,  and  18  and  Theorem  19,  the  inclusion  relations  between 
SO^  and  the  fixpoint  logics  FP  and  PFP  are  equivalent  to  open  problems  in 
complexity  theory.  The  next  result  shows  that  this  is  also  true  of  the  levels  of 
the  hierarchy  within  . 

Theorem  20.  For  every  ij  G  u ,  =  Uy’**'  if  and  only  if  Uf  =  ;  and 

^  jjhw  .j:  if  nf  . 

Finally,  we  also  observe  that  when  a  sentence  (p  of  is  translated  to 
the  ordered  quotient  structure  01/  ,  the  resulting  sentence  is,  in  fact,  in 

the  monadic  fragment  of  X*} ,  i.e.  it  only  uses  quantification  over  sets.  Writ¬ 
ing,  mon.X?^  for  the  monadic  fragment  of  E}  ,  we  can  then  extract  the  following 
result  from  the  proof  of  Theorem  20.  Note  that  this  result  is  not  about  the 
logic  SO**' .  It  is  a  result  about  the  unrestricted  second  order  logic,  although  it 
is  obtained  by  using  facts  about  SO**'  in  the  proof. 

Theorem  21.  For  any  i,j  £  u  ,  if  mon^E}  =  mon.Xy  on  ordered  sirucutures, 
ihenEf  =  E^. 

4  NP-Complete  Problems 

While  the  logic  FP  cannot  express  some  easily  computable  properties  -  such  as 
the  property  of  a  graph  having  even  cardinality,  which  is  not  even  expressible 
in  “  it  can  nevertheless  express  some  P-complete  problems  such  as  the 
path  systems  problem  and  alternating  transitive  closure  (see  [18]).  Similarly, 
Abiteboul  ei  ai  [1]  show  that  there  are  natural  PSPACE-complete  problems 
that  can  be  expressed  in  PFP.  In  this  section,  we  examine  the  expressibility  of 
NP-complete  problems  in  the  logic  Xj’**' . 

In  one  sense,  it  is  easy  to  see  that  there  are  NP-complete  problems  that  can 
be  expressed  in  Xj’**',  since  this  logic  captures  NP  on  ordered  structures  (see 
Theorem  11).  Thus,  if  we  take  any  NP-complete  problem  and  consider  the  set  of 
its  instances  with  linear  order,  we  obtain  a  problem  that  is  still  NP-complete  and 
is  expressible  in  X}’^.  For  instance,  consider  the  class  of  structures  (V,  X,  <), 
where  <  is  a  linear  order  on  V  ^  and  the  graph  {VyE)  is  Hamiltonian. 

However,  in  the  absence  of  linear  order,  many  natural  NP-complete  problems 
cannot  be  expressed  in  and  a  fortiori  not  in  Xj’*" .  Immerman  [14]  showed, 
essentially,  that  Hamil tonicity  and  clique  are  not  definable  in  .  We  present, 
as  an  example,  a  simple  proof  that  Hamiltonicity  is  not  in  . 

Example  1.  Consider  the  complete  bipartite  graph  Km,n  •  It  is  easily  verified  that 
this  graph  is  Hamiltonian  if  and  only  if  m  =  n .  An  easy  pebble  game  argument 
shows  that  Kk,k  =*  Since  Kk,k  is  Hamiltonian  and  Kk,k+i  is  not,  it 

follows  that  Hamiltonicity  is  not  in  for  any  k . 

Lovasz  and  Gacs  [20]  show  that  the  problem  of  propositional  satisfiability  (SAT) 
is  complete  for  NP  even  under  first  order  reductions.  Since  is  closed  under 

first  order  reductions,  it  follows  that  SAT  is  not  definable  in  ,  for  otherwise 
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NP  would  be  contained  in  1/^^,  which  we  know  is  not  the  case.  Similarly, 
Dahlhaus  [7]  shows  that  both  Hamiltonicity  and  clique  are  also  NP-complete 
under  first  order  reductions  and  this  provides  an  alternative  proof  that  these 
problems  are  not  expressible  in  .  Essentially,  it  is  a  consequence  of  the 
completeness  results  that  we  can  take  the  proof  that  some  query  in  NP  is  not 
in  -  say  the  even  cardinality  query  -  and  translate  it  into  a  proof  that 
Hamiltonicity  or  clique  is  not  in  . 

In  contrast,  3-colourability  is  an  NP-complete  problem  that  is  known  not  to 
be  complete  with  respect  to  first  order  reductions.  By  a  result  of  [9],  we  know 
that  the  class  of  queries  that  are  reducible  to  3-colourability  obeys  a  0-1  law. 
That  is,  for  every  Boolean  query  in  this  class,  the  proportion  of  structures  of 
size  n  that  are  instances  of  the  query  tends  to  either  0  or  1  as  n  goes  to  infinity. 
It  follows  that  straightforward  counting  arguments  such  as  in  Example  1  will 
not  suffice  to  show  that  3-colourability  is  not  expressible  in  .  By  taking  a 
different  approach,  we  are  nevertheless  able  to  show  below  that  3-colourability 
is  not  definable  in  .  This  answers  an  open  question  posed  by  Kolaitis  and 
Vardi  [18]. 

4.1  3-Colourability 

In  order  to  show  that  3-colourability  is  not  expressible  in  ,  we  adapt  a 
construction  due  to  Cai  ei  ai  [5]  which  shows  that  there  is  a  PTIME  query  that 
is  not  expressible  in  the  extension  of  FP  by  counting. 

The  crucial  idea  in  the  construction  of  Cai  ei  al.  is  to  construct  graphs  Xd , 
which  include  d  distinguished  pairs  of  nodes  (ai,6i), . .  .,(0^,6^)  with  the  fol¬ 
lowing  property: 

(*)  for  every  subset  S  of  {l,...,d}  which  is  of  even  cardinality,  there  is  an 
automorphism  of  Xd  which  exchanges  a,-  and  6,*  for  i  6  5,  while  fixing  a,- 
and  bi  for  i  ^  5 .  There  is  no  automorphism  of  Xd  that  does  this  for  a  set 
S  of  odd  cardinality. 

We  refer  to  the  pair  of  points  (a,-,  6,-)  as  the  ith  gate  of  Xd. 

We  can  construct  such  an  Xdhy  including,  in  addition  to  the  d  gates, 
nodes  vs ,  one  for  each  even  sized  subset  S  of  {1, . .  .,d}.  The  graph  Xd  then 
contains  the  edges  (ai,vs)  for  i  E  S  and  {hi,vs)  for  i  ^  5.  It  can  be  easily 
verified  that  Xd  has  property  (*). 

Examples.  The  graph  X3  is  depicted  in  Figure  1. 

Let  G  be  a  graph  such  that  every  vertex  in  G  has  degree  at  least  2.  The 
graph  X{G)  is  defined  as  follows.  Every  vertex  t;  in  G  is  replaced  by  a  copy 
of  Xd ,  where  d  is  the  degree  of  v ,  with  each  edge  incident  on  v  being  assigned 
a  gate  of  Xd.  We  denote  the  copy  of  Xd  that  replaces  u  by  X*' ,  and  its  ith 
gate  by  (aV,6V),  For  an  edge  of  G,  let  the  gates  in  JV”  and  X^  assigned 

to  this  edge  be  (aVjtJ*)  and  (aJ,6J).  The  graph  X{G)  contains  the  two  edges 
(aj*,aj)  and  (6“,6J).  We  also  define  the  graph  X(G),  which  is  obtained  from 
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Fig. 1.  Xs 


X{G)  by  “twisting”  exactly  one  edge.  That  is,  for  one  edge  (ti,  v)  of  G,  in  place 
of  the  edges  and  (65*,6p,we  include  (a}*,6p  and  (6“,ap. 

We  now  state  two  lemmas  regarding  this  construction  due  to  Cai  ei  al.  [5]. 

Lemma  22.  X{G)  and  X{G)  art  not  isomorphic. 

We  omit  a  full  proof  of  this  lemma,  oberving  only  that  any  isomorphism  from 
X{G)  to  X{G)  would,  restricted  to  some  X'’  ^  yield  an  automorphism  of  X^ 
which  exchanges  ai  and  bi  for  an  odd  number  of  gates.  This,  however,  would 
violate  the  property  (*)  of  X*' . 

Recall  that  a  separator  of  a  graph  G  =  (V^E)  is  a  set  of  nodes  U  CV  y 
such  that  the  subgraph  of  G  induced  hy  V  has  no  connected  component 
containing  more  than  \V\I2  nodes.  This  allows  us  to  formulate  the  second  lemma 
due  to  Cai  et  al.  [5]. 

Lemma  23,  If  G  has  no  separator  of  cardinality  k,  then  X{G)  X(G). 

Once  again,  we  omit  a  detailed  proof  of  this  lemma,  and  present  instead  an 
informal  description  of  Duplicator's  strategy  in  the  pebble  game.  At  any  stage 
in  the  pebble  game,  there  are  at  most  k  pebbles  on  each  of  X(G)  and  X{G). 
Consider  the  graphs  formed  by  removing  from  X(G)  and  X{G)  any  X^  that 
contains  a  pebbled  vertex.  Since  G  has  no  separator  of  cardinality  k ,  the  resuling 
graphs  each  contain  a  connected  component  that  includes  more  than  half  of  all 
the  vertices.  Duplicator’s  strategy  is  essentially  to  “hide  the  twist”  in  this  large 
component.  Clearly,  the  only  way  the  Spoiler  can  win  the  game  is  by  isolating 
the  twist,  i.e.,  by  placing  pebbles  on  two  of  the  four  vertices  in  the  two  gates 
of  the  twisted  edge  (w,  v) ,  in  such  a  way  as  to  force  Duplicator  to  interchange 
the  vertices  in  one  of  the  gates,  say  of  X“ .  She  is  then  forced  to  interchange  the 
vertices  in  another  of  the  gates  of  X^ ,  effectively  moving  the  twist  to  another 
location.  Since  after  every  move  there  is  an  unpebbled  component  containing 
more  than  half  the  vertices,  these  components  must  overlap  from  one  move  to 
the  next.  This  allows  Duplicator  to  always  keep  the  twisted  edge  in  the  large 
component,  and  therefore  Spoiler  cannot  isolate  it. 

To  adapt  the  construction  of  Cai  et  al  to  show  that  3- colour  ability  is  not 
definable  in  we  construct  a  gadget  Cd  for  every  d,  along  the  lines  of 
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Xd  above,  that  has  some  additional  properties.  We  state  the  relevant  properties 
here,  and  defer  the  explicit  construction  to  a  later  point  in  this  section. 

1.  Cd  contains  d  gates,  each  consisting  of  three  nodes  c,),  and  these 

nodes  are  connected  by  edges  to  form  a  triangle. 

2.  For  every  subset  S  of  {1, . .  .,d}  of  even  size,  there  is  an  automorphism  of 
Cd  that  exchanges  a*  and  6,-  for  i  €  5,  while  fixing  nodes  in  all  other  gates. 
There  is  no  such  automorphism  for  odd  size  sets  S . 

3.  Cd  is  3-colourable,  and  in  any  valid  3-colouring  of  Cd  all  c,-  are  assigned  the 
same  colour.  There  is  a  valid  3-colouring  of  Cd  in  which  all  a,-  are  assigned 
the  same  colour.  Finally,  this  3-colouring  is  unique  up  to  renaming  of  colours 
and  automorphisms  of  Cd . 

One  consequence  of  these  conditions  is  that  if  d  is  even,  then  in  any  3- colouring 
of  Cd  and  for  any  of  the  colours,  the  number  of  a,*  that  are  assigned  that  colour 
must  be  even. 

We  now  define,  for  every  graph  C,  the  graphs  C(C)  and  C(C)  along  the 
lines  of  X{G)  and  X(G)  above.  The  only  difference  is  that  each  edge  (u,t;) 
of  C  is  now  replaced  by  three  edges  in  C(C),  (aj‘,ap,  (6V,6p  and  (cJ'jCp. 
The  graph  C(C)  is  obtained  from  C(G)  by  replacing  exactly  one  pair  (aj*,  ap, 
(6V,6p  of  edges  by  (aV,6p,  (6}‘,ap.  The  following  lemma  is  now  immediate, 
along  the  lines  of  Lemma  23. 

Lemma  24>  If  G  has  no  separator  of  cardinality  k,  then  C(C)  =*  C(C). 

We  proceed  to  construct,  for  every  A:  G  w ,  a  graph  which  has  no  separator  of 
cardinality  Ar,  which  is  3-colourable  and  such  that  its  3-colouring  is  unique  up 
to  a  renaming  of  colours. 

Definition25.  A  triangular  mesh  of  order  n,  denoted  T„ ,  is  a  graph  with  n? 
vertices:  V(»,y),  0  <  <  n  and  the  following  edges  for  each  i  and  j : 

j+i)]:  K<- ,;•)>  »^(<+i j+i)]. 

where  the  additions  are  all  modulo  n . 

We  now  establish  the  relevant  properties  of  triangular  meshes  in  the  next 
three  lemmas. 

Lemma 26,  For  n  >  3,  Tn  has  no  separator  of  cardinality  n. 

Proof  For  each  i ,  define  the  row  Ri  to  be  the  set  of  vertices  |  0  <  j  <  n} . 

Similarly  define  the  column  Cj  =  |  0  <  z  <  n} .  Note  that  the  subgraph 

of  Tn  induced  by  each  row  and  each  column  is  a  cycle  of  length  n . 

Let  U  be  any  subset  of  the  vertices  of  Tn  such  that  \U\  =  n.  We  first  note 
that  if  U  contains  one  vertex  from  every  row  or  one  vertex  from  every  column, 
then  the  result  of  removing  the  vertices  in  U  from  Tn  is  a  connected  graph.  To 
see  this,  suppose  U  contains  one  vertex  from  every  row,  then  after  the  removal 
of  vertices  in  C/ ,  every  row  remains  connected,  since  it  is  a  cycle  with  one  vertex 
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removed,  and  since  two  successive  rows  are  connected  by  2n  edges,  the  removal 
of  one  vertex  in  each  row  will  not  disconnect  them.  Thus  such  a  U  does  not 
form  a  separator. 

Next,  we  consider  a  set  U  of  cardinality  n  which  for  some  row  and  for  some 
column  includes  at  least  two  of  its  vertices.  But  then,  there  must  be  Ri  and 
Cj  such  that  Rir\U  =  0  and  CjC\U  =  0 .  Note  that  Ri  U Cj  contains  2n  -  1 
vertices.  Now,  for  any  other  column  Ci  such  that  \CinU\  <  all  the  elements 
of  Cl  are  connected  to  by  a  path  that  does  not  include  a  vertex  in  U ,  It 
follows  that,  after  removing  U  j  at  least  n  —  2  of  the  vertices  in  at  least  half  the 
remaining  columns  are  in  the  same  connected  component  as  RiU  Cj .  Thus,  this 
component  contains  at  least  (2n  —  1)  +  (n  —  2)^/2  vertices,  which  is  more  than 
half  the  vertices  of  Tn .  Thus,  U  is  not  a  separator  of  T„ . 

Lemma  27,  If  n  is  a  multiple  ofZf  then  Tn  is  3-colourable. 

Proof  We  define  a  3-colouring  x  •  {0, 1, 2}  given  by: 

x(t^(i,;))  =  ^  i  +  j  =  l  (mod  3). 

It  is  easily  seen  that  this  is  a  valid  3-colouring  of  the  graph. 

Lemma  28,  If  n  is  a  multiple  o/3,  then  C{Tn)  is  3-colourable,  and  C{Tn)  is 
not  3-colourable. 

Proof.  To  see  that  C{Tn)  is  3-colourable,  consider  a  valid  3-colouring  of  T„ , 
X  -  Tn  -^{0,1,2}.  For  each  vertex  v  of  Tn,  we  can  colour  the  graph  C^  in  such 
a  way  that  all  c"  are  assigned  the  colour  x(^)  >  all  a*  are  assigned  the  colour 
(x(^)  +  1)  niod  3  and  all  6*  are  assigned  the  colour  (x(v)  +  2)  mod  3.  It  can 
then  be  easily  verified  that  this  results  in  a  valid  3-colouring  of  C(Tn). 

To  show  that  C{Tn)  is  not  3-colourable,  we  make  the  following  observations. 
First,  the  edges  of  Tn  can  be  partitioned  into  3n  sets;  each  of  which  forms  a 
cycle  of  length  n .  These  are  given  by  the  n  rows  Ri  =  |  0  <  j  <  n} ,  the  n 

columns  Cj  =  |  0  <  i  <  n}  and  the  n  diagonals  Dk  =  {(i,  Ar  -f-  i  mod  n)  | 

0  <  i  <  n}.  Each  vertex  then  appears  in  exactly  three  such  cycles.  Secondly, 
given  any  valid  3-colouring  of  C{Tn),  we  obtain  a  valid  3-colouring  of  Tn  by 
assigning  the  colour  of  c"  to  the  vertex  v.  In  particular,  this  implies  that  if 
we  consider  the  colours  of  in  an  n-cycle,  then  they  must  strictly  alternate 
among  the  three  colours.  Thirdly,  given  a  gadget  C^  in  C(Tn),  we  pair  off  the 
gates  of  C'^  into  three  pairs,  the  horizontal,  vertical  and  diagonal,  according  to 
which  n-cycle  they  appear  in.  Given  a  3-colouring  of  if  the  two  aV  in 

one  such  pair  are  of  different  colours,  we  say  that  C^  makes  a  switch  in  the 
corresponding  n-cycle.  By  property  (3)  of  the  gadgets  C" ,  it  follows  that  each 
one  of  them  makes  an  even  number  of  switches.  Finally,  we  note  that  for  a  valid 
3-colouring  of  C{Tn),  each  n-cycle  must  contain  an  even  number  of  switches, 
except  the  unique  n-cycle  containing  the  twisted  edge,  which  must  contain  an 
odd  number  of  switches.  It  can  be  easily  verified  that  these  last  two  requirements 
lead  to  a  contradiction. 
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Finally,  we  give  an  explicit  construction  of  the  gadget  Cd  having  the  prop¬ 
erties  (l)-(3)  listed  above.  Note  that,  since  the  graph  Tn  is  regular  of  degree 
6,  it  would  suffice  to  give  a  construction  of  Ce .  However,  we  present  a  general 
purpose  construction. 

The  graph  Cd  has  16d  nodes  altogether.  It  contains  a  spine  of  3d  nodes, 
So,  •  •  • ,  with  edges  (s,-,  Sf+i)  and  (s,,  5,^2)  for  all  i.  Here,  and  in  the  rest 
of  this  section,  addition  in  the  subscripts  is  understood  to  be  modulo  3d.  That 
is,  the  spine  consists  of  a  cycle  of  length  3d,  along  with  all  its  chords  of  length 
two.  Thus,  in  any  3-colouring  of  Cd,  the  nodes  s,-  and  must  be  assigned  the 
same  colour,  for  all  t,  i.e.,  the  colours  along  the  spine  strictly  alternate  among 
all  three  colours. 

Next,  attached  to  each  s,- ,  there  are  two  additional  vertices  /,•  and  r* ,  which 
are  also  attached  to  each  other.  Morover,  if  i  =  1  (mod  3)  or  i  =  2  (mod  3) , 
then  li  (resp.  r,*)  is  connected  by  an  edge  to  (resp.  rj+i).  A  part  of  the 
spine  is  depicted  in  Figure  2.  For  clarity,  the  chords  of  length  two  along  the 
spine  are  omitted.  It  can  be  verified  from  Figure  2  that  given  a  3-colouring  of 
the  spine,  the  colouring  of  the  U  and  ri  is  determined  up  to  automorphism. 


Fig.  2.  A  portion  of  the  spine  of  Cd . 


Finally,  in  the  gap  between  S3i+2  and  S3(j+i),  we  place  the  ith  gate  of  Cj, 
by  attaching  the  pair  (a,-,  6,-)  to  the  pairs  (/3t+2 ,  ^3»+2)  and  (/3(i+i),  J’3(j+i))  t>y 
means  of  the  gadget  X3  of  Example  2.  To  ensure  uniqueness  of  3-colouring,  we 
also  connect  Uj  and  6,*  by  edges  to  S3, -4. 2 .  An  example  is  depicted  in  Figure  3, 
where,  for  clarity  the  edges  that  are  part  of  X3  are  indicated  by  dashed  lines. 
It  can  be  verified  from  Figure  3  that,  if  the  colouring  of  /3i+2,  ?’3»+2,  ^3(t+i)  and 
^3(*+i)  is  fixed,  this  also  fixes  the  colouring  of  a,-  and  6,- . 

If  we  now  consider  an  automorphism  that  exchanges  a,-  and  6,- ,  it  must  also 
exchange  exactly  one  of  the  pairs  (/3i+2,  ^31+2)  or  (/3(,+i),  r3(,.{.i)).  Assuming, 
without  loss  of  generality,  that  it  is  the  latter,  the  automorphism  must  also 
exchange  (/3,.j.4,  r3j+4)  and  (/si+s,  T’st+s)  which  takes  us  to  the  next  gate,  where 
we  can  choose  to  either  exchange  and  6,+i  or  to  continue  to  exchange 

nodes  /  and  r  further  along  the  spine.  In  any  case,  we  must  exchange  aj  and 
bj  for  some  j  ^  ,  which  gives  us  the  required  properties  of  the  gadget  Cd- 

Thus,  having  established  that  there  exists  gadgets  Cd  with  the  properties 
(l)-(3),  the  main  theorem  of  this  section  follows  as  a  consequence  of  Lem¬ 
mas  24,  26  and  28. 
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Fig.  3.  A  gate  of  Cd 


Theorem  29,  S-colourahiliiy  is  not  expressible  in 

Remark.  The  proof  of  Theorem  29  given  above  can  be  adapted  to  show  that 
3"Colourability  is  not  even  definable  in  the  extension  of  with  counting 
quantifiers.  To  see  this,  note  that  if  we  add  two  constants  c  and  d  to  our  sig¬ 
nature  and  interpret  them  in  a  triangular  mesh  Tn  by  two  adjacent  vertices  in 
the  same  row,  then  there  is  an  FP  formula  that  defines  a  linear  order  in  T„  .  It 
follows  that  if  c  and  d  are  interpreted  by  vertices  in  adjacent  gadgets  in  C{Tn) 
and  C{Tn)f  then  the  sizes  of  the  =*  equivalence  classes  in  these  structures  are 
bounded.  Moreover,  we  Can  now  even  remove  the  constants  from  our  signature 
simply  by  distinguishing  the  two  gadgets  in  some  identifiable  way,  say  an  extra 
vertex  that  does  not  interfere  with  the  relevant  properties  of  the  gadget.  How¬ 
ever,  when  the  sizes  of  the  equivalence  classes  are  bounded,  then  counting 
quantifiers  do  not  add  to  the  expressive  power  of  (for  details  see  [5]). 

4.2  NFA  Inequivalence 

In  this  section  we  examine  examples  of  natural  NP-complete  problems  that  are 
expressible  in  the  logic  .  The  examples  are  special  cases  of  the  problem  of 
NFA  inequivalence,  that  is  the  problem  of  deciding,  given  two  non-deterministic 
finite  automata,  whether  or  not  they  accept  distinct  languages.  This  problem  is 
PSPACE-complete,  and  was  shown  by  Abiteboul  ei  al  [1]  to  be  expressible  in 
PFP.  Two  restrictions  of  this  problem  that  are  known  the  be  NP-complete  are 
the  restriction  to  a  finite  language  and  the  restriction  to  a  unary  alphabet  (see 
[12]).  Both  of  these  restrictions  are  definable  in  .  We  examine  the  second 
one  in  some  detail. 

The  problem  of  determining  whether  two  NFAs  over  a  unary  alphabet  are 
inequivalent,  which  we  denote  UNI  (for  unary  NFA  inequivalence),  can  also  be 
formulated  as  a  problem  on  graphs,  as  follows.  Given 

N  =  (V,  A,  so,fo,sij<i)i 
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where  {V,A)  is  a  directed  graph  and  so,si,to,ii  6  V  are  distinguished  vertices, 
are  the  two  sets 

pQ  —  {p£u)\  there  is  a  path  of  length  p  from  $q  to  to} 

Pi  =  {p  e  w  I  there  is  a  path  of  length  p  from  si  to  ti} 

distinct? 

To  see  that  this  problem  is  definable  in  ,  we  first  observe  that  if  there  is 
a  p  €  w  that  distinguishes  the  two  sets  Pq  and  Pi  in  a  structure  N ,  then  there 
is  such  a  p  <  where  t  is  the  number  of  distinct  equivalence  classes  in 
N .  This  is  because,  for  every  p,  there  is  a  formula  y)  of  which  asserts 

that  there  is  a  path  of  length  p  from  x  to  y.  Thus,  if  we  consider  the  set  of 

pairs  (a:,  y)  such  that  there  is  a  path  of  length  p  from  so  to  x  if,  and  only  if, 
there  is  a  path  of  length  p  from  si  to  y,  then  this  set  is  -^-closed.  Moreover,  it 
can  also  be  easily  verified  that  is  definable  from  .  Since  there  are  only 

2^  distinct  -closed  sets,  it  follows  that  for  p  >  2*  the  sequence  of  sets  must 
repeat  itself.  Thus,  if  the  pair  (iojti)  appears  in  the  set  E^  for  all  p  <  2* ,  then 
it  appears  in  all  £?^,p  €  which  means  that  the  sets  Pq  and  Pi  are  identical. 
However,  if  there  is  a  p  <  2*  such  that  (<o,<i)  ^  j  then  this  p  witnesses  that 
Po  and  Pi  are  distinct. 

Next,  we  note  that  any  number  p  <  2*  can  be  represented  by  a  pair  of 
relations  where  is  the  ordering  of  -equivalence  classes  given  by 

Theorem  8,  and  P  is  a  -closed  relation.  This  is  done  by  interpreting  the  pair 
(<^,P)  as  a  binary  string  of  length  with  a  1  for  each  —^-equivalence  class 
that  is  in  R  and  a  0  for  each  class  that  is  not  in  R.  Finally,  we  note  that,  given 
the  pair  (<^,  R) ,  we  can  write  a  sentence  R)  of  FP  which  asserts  that  the 

number  represented  by  (<^,P)  distinguishes  the  sets  Po  and  Pi.  We  will  not 
write  down  (p  explicitly,  noting  only  that  we  can  write  an  inductive  definition  of 
a  5 -ary  relation  S  such  that  5(xi,a;2,y)  holds  if  and  only  if,  whenever  y  is  in 
the  eth  -equivalence  class,  there  is  a  path  of  length  p,-  from  xi  to  X2 ,  where 
Pi'  is  the  number  represented  by  the  first  i  bits  of  (<^,P). 

Now,  it  is  clear  that  the  sentence  3^03^R(p{0jR)  expresses  the  problem 
UNI.  Moreover,  it  follows  from  Corollary  16,  that  this  is  equivalent  to  a  sentence 
of  .  This  enables  us  to  establish  the  following  theorem. 

Theorem 30.  PTIME  =  NP  if,  and  only  if,  UNI  G  FP. 

Proof  In  one  direction,  if  UNI  is  definable  in  FP,  it  is  solvable  in  polynomial 
time.  Since  the  problem  is  NP-complete,  this  means  that  PTIME  =  NP . 

In  the  other  direction,  if  PTIME  =  NP,  then  by  Theorem  17,  =  FP. 

But,  since  UNI  €  EI^^  it  follows  that  UNI  G  FP. 


5  Conclusion 

A  great  deal  of  research  in  finite  model  theory  has  been  inspired  by  the  discov¬ 
ery  of  the  close  connection  between  logical  expressibility  and  coniputational  com¬ 
plexity.  This  discovery  raises  the  possibility  of  applying  model-theoretic  methods 
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to  attack  outstanding  open  problems  in  complexity  theory.  Unfortunately,  most 
of  the  results  and  methods  that  have  been  developed  in  the  study  of  infinite 
models  do  not  apply  when  only  finite  models  are  considered.  To  a  large  extent, 
the  classical  subject  of  model  theory  can  be  seen  to  be  the  study  of  the  relation 
of  elementary  equivalence.  However,  this  equivalence  relation  turns  out  to  be  of 
limited  interest  in  the  logical  study  of  finite  models,  since  it  is  identical  with 
isomorphism.  Recent  work  has  shown,  nonetheless,  that  there  is  an  equivalence 
relation  (or  rather  a  countable  collection  of  such  relations),  namely  which 
has  a  close  connection  with  logical  definability  and  which  is  non-trivial  on  finite 
models.  Moreover,  as  this  paper  illustrates,  outstanding  questions  in  complexity 
theory  can  be  reformulated  in  a  context  where  this  equivalence  relation  corre> 
sponds  with  the  notion  of  definability.  This  further  underlines  the  need  to  study 
the  model  theory  of  finite  variable  logics. 
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Abstract:  It  is  well-known  that  the  complexity  class  NP  coincides  with  the 
class  of  problems  expressible  in  existential  second-order  logic  (i7j).  Monadic  NP 
is  the  class  of  problems  expressible  in  monadic  ,  i.e.,  Sl  with  the  restriction 
that  the  second-order  quantifiers  range  only  over  sets  (as  opposed  to  ranging 
over,  say,  binary  relations).  The  author  introduced  a  type  of  Ehrenfeucht-Frai'sse 
game,  called  the  monadic  NP  game,  to  prove  that  connectivity  is  not  in  monadic 
NP.  Later,  Ajtai  and  the  author  introduced  another  type  of  monadic  NP  game 
(the  “Ajtai-Fagin  monadic  NP  game”)  to  prove  that  directed  reachability  is  not 
in  monadic  NP.  Both  games  have  two  players  (the  spoiler  and  the  duplicator), 
and  involve  coloring  steps  (where  the  players  color  nodes  of  the  graphs)  and 
selection  steps  (where  the  players  select  nodes  of  the  graphs,  round  by  round). 
It  is  known  that  the  original  game  and  the  Ajtai-Fagin  game  are  equivalent,  in 
the  sense  that  both  characterize  monadic  NP.  Thus,  the  duplicator  has  a  winning 
strategy  in  the  original  game  for  every  choice  of  parameters  (number  of  colors 
and  number  of  rounds)  if  and  only  if  the  duplicator  has  a  winning  strategy  in  the 
Ajtai-Fagin  game  for  every  choice  of  parameters.  In  this  paper,  we  investigate 
the  relationship  between  these  games  at  a  finer  level.  We  show  that  in  one  sense, 
even  at  a  finer  level,  Ajtai-Fagin  monadic  NP  games  are  no  stronger  than  the 
original  monadic  NP  games.  Specifically,  we  show  that  the  families  of  graphs 
used  in  the  Ajtai-Fagin  game  to  prove  that  a  problem  is  not  in  monadic  NP  can 
in  principle  be  used  in  the  original  game  to  prove  the  same  result  (where  for 
a  given  choice  of  parameters,  bigger  graphs  of  the  same  type  are  used  for  the 
original  game  than  for  the  Ajtai-Fagin  game).  This  answers  an  open  question  of 
Ajtai  and  the  author.  We  also  show  that  in  another  sense,  Ajtai-Fagin  games  are 
stronger,  in  that  there  are  situations  where  the  spoiler  requires  more  resources 
(colors)  to  win  the  Ajtai-Fagin  game  than  the  original  game,  when  the  choices 
of  graphs  are  fixed.  Our  analysis  gives  a  nonelementary  upper  bound,  which  we 
conjecture  to  be  optimal,  on  the  number  of  extra  colors  that  are  required  for  the 
spoiler  to  win  the  Ajtai-Fagin  game  than  the  original  game. 
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1  Introduction 

The  computational  complexity  of  a  problem  is  the  amount  of  resources,  such  as 
time  or  space,  required  by  a  machine  that  solves  the  problem.  The  descriptive 
complexity  of  a  problem  is  the  complexity  of  describing  the  problem  in  some 
logical  formalism  [Imm89].  There  is  an  intimate  connection  between  the  descrip¬ 
tive  complexity  and  the  computational  complexity.  In  particular  [Fag74],  the 
complexity  class  NP  coincides  with  the  class  of  properties  of  finite  structures 
expressible  in  existential  second-order  logic,  otherwise  known  as  Tf.  A  conse¬ 
quence  of  this  result  is  that  NP=co-NP  if  and  only  if  existential  and  universal 
second-order  logic  have  the  same  expressive  power  over  finite  structures,  i.e.,  if 
and  only  if  El  —  ITl. 

One  way  of  attacking  these  difficult  questions  is  to  restrict  the  classes  under 
consideration.  Instead  of  considering  El  (=NP)  and  JUl  (=co-NP)  in  their  full 
generality,  we  could  consider  the  monadic  restriction  of  these  classes,  i.e.,  the 
restriction  obtained  by  allowing  second-order  quantification  only  over  sets  (as 
opposed  to  quantification  over,  say,  binary  relations).  Following  Fagin,  Stock- 
meyer,  and  Vardi  [FSV93],  we  refer  to  the  restricted  classes  as  monadic  NP 
(resp.,  monadic  co-NP).  It  should  be  noted  that,  in  spite  of  its  severely  restricted 
syntax,  monadic  NP  does  contain  NP-complete  problems,  such  as  3-colorability 
and  satisfiability.  The  hope  is  that  the  restriction  to  the  monadic  classes  will 
yield  more  tractable  questions  and  will  serve  as  a  training  ground  for  attacking 
the  problems  in  their  full  generality. 

As  a  first  step  in  this  program,  the  author  [Fag75]  separated  monadic  NP 
from  monadic  co-NP.  Specifically,  it  was  shown  that  connectivity  of  finite  graphs 
is  not  in  monadic  NP,  although  it  is  easy  to  see  that  it  is  in  monadic  co-NP. 
The  proof  that  connectivity  is  not  in  monadic  NP  makes  use  of  a  certain  type 
of  Ehrenfeucht-Fraisse  game  on  graphs  played  between  two  players,  called  the 
spoiler  and  the  duplicator.  The  game  involves  coloring  steps  (where  the  players 
color  nodes  of  the  graphs)  and  selection  steps  (where  the  players  select  nodes 
of  the  graphs,  round  by  round).  We  call  this  game  the  (original)  monadic  ’NP 
game.  In  this  game,  the  duplicator  selects  two  graphs  Go  and  Gi,  where  Go 
is  connected  and  Gi  is  not.  The  spoiler  then  colors  Go,  and  the  duplicator 
colors  Gi .  They  then  play  a  first-order  Ehrenfeucht-Fraisse  game  on  these  colored 
graphs,  where,  as  usual,  the  spoiler  tries  to  expose  differences  in  the  graphs,  and 
the  duplicator  tries  to  cover  up  these  differences.  A  necessary  and  sufficient 
condition  for  proving  that  connectivity  is  not  in  monadic  NP  is  to  show  that  for 
each  choice  of  parameters  (number  of  colors  and  number  of  first-order  rounds), 
there  are  graphs  Go  and  Gi  on  which  the  duplicator  has  a  winning  strategy.  By 
showing  that,  indeed,  the  duplicator  has  a  winning  strategy,  the  author  showed 
that  connectivity  is  not  in  monadic  NP. 

Later,  Ajtai  and  the  author  [AF90]  continued  this  program  by  showing  that 
(s,t)-connectivity  of  directed  graphs  (otherwise  known  as  directed  reachability) 
is  not  in  monadic  NP.  They  made  use  of  a  modified  game,  which  is  now  often 
referred  to  as  the  Ajtai-Fagin  monadic  NP  game.  Here  the  duplicator  selects  a 
graph  Go  that  is  (s,t)-connected,  and  the  spoiler  colors  Go-  Then  the  duplica- 
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tor  selects  and  colors  a  graph  G\  that  is  not  (5,t)-connected.  The  game  again 
concludes  with  a  first-order  game.  The  difference  between  the  Ajtai-Fagin  game 
and  the  original  game  is  that  in  the  Ajtai-Fagin  game,  the  spoiler  must  commit 
himself  to  a  coloring  of  Go  before  seeing  Gi.  Putting  it  another  way,  the  dupli¬ 
cator  can  wait  to  decide  on  his  choice  of  Gi  until  he  sees  how  the  spoiler  colors 
Gq.  Because  the  change  in  rules  between  the  original  game  and  the  Ajtai-Fagin 
game  favors  the  duplicator,  on  the  face  of  it  the  Ajtai-Fagin  game  is  “easier  for 
the  duplicator  to  win”,  which  makes  it  easier  to  prove  that  the  duplicator  has 
a  winning  strategy.  In  fact,  Ajtai  and  the  author  introduced  their  variation  on 
the  original  game  because  they  did  not  see  how  to  prove  that  the  duplicator  has 
a  winning  strategy  in  the  original  game.  However,  they  were  able  to  prove  that 
the  duplicator  has  a  winning  strategy  in  their  variation  of  the  game.  Since  the 
duplicator  has  a  winning  strategy,  directed  reachability  is  not  in  monadic  NP. 

There  is  some  mystery  about  the  relationship  between  the  Ajtai-Fagin  game 
and  the  original  game.  On  the  one  hand,  the  two  games  are  equivalent,  in  the 
sense  that  in  both  cases,  the  existence  of  a  winning  strategy  for  the  spoiler  is 
a  necessary  and  sufficient  condition  for  a  class  to  be  in  monadic  NP.  Thus,  in 
both  cases,  showing  that  a  problem  is  not  in  monadic  NP  corresponds  precisely 
to  showing  that  the  duplicator  has  a  winning  strategy.  On  the  other  hand,  as 
we  noted,  the  Ajtai-Fagin  game  seems  intuitively  to  be  easier  for  the  duplicator 
to  win.  Because  of  the  fundamental  role  of  Ehrenfeucht-Fraisse  games  as  tools 
in  descriptive  complexity,  it  is  important  to  understand  better  the  difference  in 
power  of  the  Ajtai-Fagin  game  and  the  original  game.  In  this  paper,  we  explore 
this  difference. 

In  both  games,  a  class  S  is  given  (such  as  the  class  of  connected  graphs, 
or  the  class  of  (s,  t)-connected  graphs).  Then  various  graphs  are  selected  and 
colored  by  the  players,  and  a  first-order  game  is  played  on  these  colored  graphs. 
The  equivalence  of  the  games  corresponds  to  the  fact  that  for  each  class  «S, 
the  duplicator  has  a  winning  strategy  in  the  original  game  for  every  choice  of 
parameters  (number  of  colors  and  number  of  rounds)  if  and  only  if  the  duplicator 
has  a  winning  strategy  in  the  Ajtai-Fagin  game  for  every  choice  of  parameters. 

In  this  paper,  we  investigate  the  relationship  between  the  original  game  and 
the  Ajtai-Fagin  game  at  a  finer  level.  We  show  that  in  one  sense,  even  at  a  finer 
level,  Ajtai-Fagin  monadic  NP  games  are  no  stronger  than  the  original  monadic 
NP  games.  This  sense  corresponds  to  fact  that  in  a  game-theoretic  proof  that 
a  class  is  not  in  monadic  NP,  the  same  families  of  graphs  can  be  used  in  the 
original  game  as  in  the  Ajtai-Fagin  game.  We  also  show  that  in  another  sense, 
Ajtai-Fagin  games  are  stronger,  in  that  there  are  situations  where  the  spoiler 
requires  more  resources  (colors)  to  win  the  Ajtai-Fagin  game  than  the  original 
game, 'when  the  choices  of  graphs  are  fixed.  We  now  explain  the  details  a  little 
more. 

In  a  game-theoretic  proof  that  a  specific  class  of  graphs  is  not  in  monadic  NP, 
the  duplicator  inevitably  restricts  himself  to  selecting  graphs  only  of  a  certain 
type.  For  example,  in  the  case  of  connectivity  [Fag75],  the  graph  Go  is  a  cycle, 
and  Gi  is  a  disjoint  union  of  two  cycles.  In  the  case  of  directed  reachability 
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[AF90],  the  graph  Go  is  a  path  from  5  to  t  along  with  certain  backedges,  and  Gi 
is  the  result  of  deleting  one  forward  edge  from  Go-  We  show  that  the  family  of 
graphs  used  in  the  Ajtai-Fagin  game  to  prove  that  a  problem  is  not  in  monadic 
NP  can  in  principle  be  used  in  the  original  game  to  prove  the  same  result  (where 
for  a  given  choice  of  parameters,  bigger  graphs  of  the  same  type  are  used  for 
the  original  game  than  for  the  Ajtai-Fagin  game).  For  example,  in  the  case  of 
directed  reachability,  we  show  that  for  every  choice  of  parameters,  the  duplicator 
has  a  winning  strategy  in  the  (original)  monadic  NP  game  where  the  graph  Go  is 
a  path  from  s  to  f  along  with  certain  backedges,  and  Gi  is  the  result  of  deleting 
one  forward  edge  from  Go  -  This  answers  an  open  question  of  Ajtai  and  the  author 
[AF90]. 

How  do  we  obtain  this  result?  First,  we  generalize  the  framework  of  the 
games.  Rather  than  saying  that  Hie  duplicator  selects  Go  from  a  class  «S,  and 
selects  Gi  from  the  complement  5,  we  instead  consider  a  more  general  game, 
where  the  duplicator  selects  Go  from  a  class  ^0  >  and  Gi  from  a  class  Qi .  There 
are  once  again  two  versions,  one  corresponding  to  the  original  game,  and  one 
to  the  Ajtai-Fagin  game.  In  the  first  version  of  this  new  game,  the  duplicator 
selects  Gi  before  the  spoiler  has  colored  Go;  in  the  Ajtai-Fagin  version,  the 
duplicator  selects  Gi  after  the  spoiler  has  colored  Go-  We  show  that  for  each 
choice  of  the  number  c  of  colors  and  the  number  r  of  rounds,  there  are  c'  and 
r'  such  that  for  every  choice  of  Qo  and  Qi  where  the  duplicator  has  a  winning 
strategy  in  the  Ajtai-Fagin  version  of  the  new  game  with  parameters  c'  and  r\ 
the  duplicator  also  has  a  winning  strategy  in  the  first  version  of  this  game  with 
parameters  c  and  r  (in  fact,  we  can  take  r'  =  r).  This  result  tells  us  that  the 
same  families  of  graphs  can  be  used  in  the  original  game  as  in  the  Ajtai-Fagin 
game  (such  as  to  prove  that  a  class  is  not  in  monadic  NP).  Intuitively,  for  a 
given  choice  of  c,  r,  we  use  bigger  graphs  in  the  original  (c,  r)-game  than  in  the 
Ajtai-Fagin  (c,7’)-game,  since  in  the  original  game  we  use  QoyOi  that  correspond 
to  the  Ajtai-Fagin  game  with  more  colors  (since  c'  >  c). 

We  now  consider  a  sense  in  which  Ajtai-Fagin  games  are  stronger.  Here,  we 
investigate  the  resources  involved  in  the  games.  Specifically,  we  consider  the 
number  of  colors  required  for  the  spoiler  to  win  when  the  choices  of  graphs  are 
fixed.  Since  the  spoiler  is  trying  to  expose  differences  between  Go  and  Gi,  and 
the  duplicator  is  trying  to  cover  up  these  differences,  it  helps  the  spoiler  for  there 
to  be  more  colors.  We  show  that  there  are  situations  where  the  spoiler  requires 
strictly  more  colors  to  win  the  Ajtai-Fagin  game  than  the  original  game.  Thus, 
in  such  situations,  it  is  indeed  true,  in  a  precise  sense,  that  it  is  easier  for  the 
duplicator  to  win  the  Ajtai-Fagin  game  than  the  original  game. 

Our  analysis  gives  a  nonelementary  upper  bound  on  the  number  of  extra 
colors  that  are  required  for  the  spoiler  to  win  the  Ajtai-Fagin  game  than  the 
original  game.  We  conjecture  that  there  is  also  a  nonelementary  lower  bound. 

This  version  does  not  contain  proofs,  which  the  interested  reader  can  find  in 
[Fag94]. 
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2  Definitions  and  conventions 

We  begin  with  a  few  conventions.  For  convenience,  we  shall  usually  discuss  only 
graphs  (usually  directed  graphs,  sometimes  with  distinguished  points  s  and  t), 
but  everything  we  say  can  be  generalized  to  arbitrary  structures.  We  are  also 
interested  in  “colored  graphs”,  where  each  vertex  has  some  color.  (When  there 
are  2^  possible  colors  for  some  k,  it  is  often  convenient  to  think  of  the  coloring 
of  a  point  as  a  description  of  which  of  k  possible  unary  relations  the  point  is 
a  member  of.)  We  assume  throughout  this  paper  that  we  are  restricting  our 
attention  to  finite  graphs  (and  so  are  doing  finite-model  theory),  although  all  of 
the  results  hold  also  without  this  assumption. 

A  sentence  is  a  sentence  of  the  form  3Ai...3A;b^,  where  ^jj  is  first-order 
and  where  the  A^’s  are  relation  symbols.  As  an  example,  we  now  construct  a 
sentence  that  says  that  a  graph  (with  edge  relation  denoted  by  F)  is  3-colorable. 
In  this  sentence,  the  three  colors  are  represented  by  the  monadic  relation  symbols 
Ai,  A2,  and  A3.  Let  -01  say  “Each  point  has  exactly  one  color”.  Thus,  is 

Va;((Aia;  A  -1A2X  A  -lAsx)  V  (-lAirr  A  A2X  A  -^Asx)  V  (-.An  A  -nA2a;  A  Asa;)). 

Let  02  say  “No  two  points  with  the  same  color  are  connected  by  an  edge”.  Thus, 
02  is 

VxVy  ((An  A  Aiy  =4^  -nPxy)  A  (A2a;  A  A22/  =>  -^Fxy)  A  (A3X  A  Asy  =>  ->Fxy)) . 

The  F}  sentence  3Ai3A23A3(0i  A  02)  then  says  “The  graph  is  3-colorable.” 

A  sentence  3Ai...3Aa;0,  where  'll;  is  first-order,  is  said  to  be  monadic  if 
each  of  the  Aj’s  is  unary,  that  is,  the  existential  second-order  quantifiers  quantify 
only  over  sets.  A  class  S  of  graphs  is  said  to  be  {monadic)  ^7^  if  it  is  the  class 
of  all  graphs  that  obey  some  fixed  (monadic)  El  sentence.  One  reason  that  El 
classes  are  of  great  interest  is  the  result  [Fag74]  that  the  collection  of  El  classes 
coincides  with  the  complexity  class  NP.  For  this  reason,  as  we  noted  earlier,  we 
follow  Fagin,  Stockmeyer,  and  Vardi  [FSV93]  by  referring  to  the  collection  of 
monadic  El  classes  as  monadic  NP.  We  often  refer  to  a  class  of  graphs  by  a 
defining  property,  for  example,  3-colorability.  As  we  saw  above,  3-colorability  is 
in  monadic  NP. 


3  Ehrenfeucht-Fraisse  Games 

Among  the  few  tools  of  model  theory  that  “survive”  when  we  restrict  our  at¬ 
tention  to  finite  structures  are  Ehrenfeucht-Fraisse  games  [EhrGl,  Fra54].  We 
begin  with  an  informal  definition  of  an  r-round  first-order  Ehrenfeucht-Fraisse 
game  (where  r  is  a  positive  integer),  which  we  shall  call  an  r-game  for  short. 
It  is  straightforward  to  give  a  formal  definition,  but  we  shall  not  do  so.  As  we 
mentioned  earlier,  for  ease  in  description,  we  shall  restrict  our  attention  to  col¬ 
ored  graphs.  There  are  two  players,  called  the  spoiler  and  the  duplicator,  and  two 
colored  graphs,  Gq  and  Gi.  In  the  first  round,  the  spoiler  selects  a  point  in  one 


419 


of  the  two  colored  graphs,  and  the  duplicator  selects  a  point  in  the  other  colored 
graph.  Let  ai  be  the  point  selected  in  Go,  and  let  hi  be  the  point  selected  in  Gi. 
Then  the  second  round  begins,  and  again,  the  spoiler  selects  a  point  in  one  of 
the  two  colored  graphs,  and  the  duplicator  selects  a  point  in  the  other  colored 
graph.  Let  02  be  the  point  selected  in  Go,  and  let  62  be  the  point  selected  in 
Gi-  This  continues  for  t  rounds.  The  duplicator  wins  if  the  colored  subgraph  of 
Go  induced  by  (ai, . . .  ,ar)  is  isomorphic  to  the  colored  subgraph  of  Gi  induced 
by  (^^1, •  •  •  ,&r})  under  the  function  that  maps  Oj  onto  6i  for  1  <  i  <  r.  That  is, 
for  the  duplicator  to  win,  (a)  ai  =  aj  iff  hi  =  for  each  i,j;  (b)  {ai,aj)  is  an 
edge  in  Go  iff  (h^hj)  is  an  edge  in  Gi,  for  each  i,j;  and  (c)  ai  has  the  same 
color  as  hi,  for  each  i.  Otherwise,  the  spoiler  wins.  We  say  that  the  spoiler  or  the 
duplicator  has  a  winning  strategy  if  he  can  guarantee  that  he  will  win,  no  matter 
how  the  other  player  plays.  Since  the  game  is  finite,  and  there  are  no  ties,  the 
spoiler  has  a  winning  strategy  iff  the  duplicator  does  not.  If  the  duplicator  has 
a  winning  strategy,  then  we  write  Go  Gi .  In  this  case,  intuitively,  Go  and  Gi 
are  indistinguishable  by  an  r-game. 

We  now  discuss  a  more  complicated  game,  which  is  a  c-color,  r~round,  monadic 
NP  game,  and  which  we  shall  call  a  {c,r)-game  for  short.  This  game  was  intro¬ 
duced  in  [Fag75]  to  prove  that  connectivity  is  not  in  monadic  NP.  We  start  with 
two  graphs  Go  and  Gi  (in  this  case,  not  colored).  Let  G  be  a  set  of  c  distinct 
colors.  The  spoiler  first  colors  each  of  the  points  of  Go,  using  the  colors  in  G,  and 
then  the  duplicator  colors  each  of  the  points  of  Gi,  using  the  colors  in  G.  Note 
that  there  is  an  asymmetry  in  the  two  graphs  in  the  rules  of  the  game,  in  that 
the  spoiler  must  color  the  points  of  Go,  not  Gi.  The  game  then  concludes  with 
an  r-game.  The  duplicator  now  wins  if  the  colored  subgraph  of  Go  induced  by 
(oi, . . . , ttr)  is  isomorphic  to  the  colored  subgraph  of  Gi  induced  by  (61, , 6^), 
under  the  function  that  maps  Oi  onto  hi  foi  1  <i  <r. 

Theorem  1.  [Fag75]  Let  S  he  a  class  of  graphs.  S  is  in  monadic  NP  iff  there 
are  c,r  such  that  whenever  Go  €  «S  and  Gi  G  S,  then  the  spoiler  has  a  winning 
strategy  in  the  (c^r)-game  over  Go, Gi. 

In  [Fag75]  it  is  shown  that  given  c  and  r,  there  is  a  graph  Go  that  is  a  cycle, 
and  a  graph  Gi  that  is  the  disjoint  union  of  two  cycles,  such  that  the  duplicator 
has  a  winning  strategy  in  the  (c,  r)-game  over  Go,  Gi .  Since  Go  is  connected  and 
Gi  is  not,  it  follows  from  Theorem  1  that  connectivity  is  not  in  monadic  NP. 

In  addition  to  considering  games  over  pairs  Go,Gi  of  graphs,  Ajtai  and  the 
author  [AF90]  found  it  convenient,  for  reasons  we  shall  see  shortly,  to  consider 
games  over  a  class  S.  The  rules  of  a  (c,  r)-game  over  S  are  as  follows. 

1.  The  duplicator  selects  a  member  of  5  to  be  Go. 

2.  The  duplicator  selects  a  member  of  S  to  be  Gi. 

3.  The  spoiler  colors  Go  with  the  c  colors. 

4.  The  duplicator  colors  Gi  with  the  c  colors. 

5.  The  spoiler  and  duplicator  play  an  r-game  on  the  colored  Go,Gi. 

The  next  theorem  follows  easily  from  Theorem  1. 
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Theorem  2.  Let  S  be  a  class  of  graphs.  S  is  in  monadic  NP  iff  there  are  c,r 
such  that  the  spoiler  has  a  winning  strategy  in  the  {c,r)~game  over  S. 

We  now  explain  why  Ajtai  and  the  author  allowed  Go  and  Gi  to  be  selected 
by  the  duplicator,  rather  than  inputs  to  the  game.  A  directed  graph  with  dis¬ 
tinguished  points  is  said  to  be  {Syt)-connected  if  there  is  a  directed  path  in 
the  graph  from  s  to  t.  Ajtai  and  the  author  wished  to  prove  that  directed  (5,  t)- 
connectivity  (also  known  as  directed  reachability)  is  not  in  monadic  NP,  but  they 
did  not  see  how  to  prove  this  by  using  (c,  r’)-games.  By  considering  the  choice 
of  C?o  and  Gi  to  be  moves  of  the  duplicator,  rather  than  inputs  to  the  game, 
they  were  able  to  define  a  variation  of  (c,  r)-games,  in  which  the  choice  of  Gi 
by  the  duplicator  is  delayed  until  after  the  spoiler  has  colored  Gq  .  They  success¬ 
fully  used  the  new  game  to  prove  the  desired  result  (that  directed  reachability 
is  not  in  monadic  NP).  Their  new  game,  which  is  usually  called  the  Ajtai-Fagin 
(c,  r)-game,  is,  on  the  face  of  it,  easier  for  the  duplicator  to  win.  The  rules  of  the 
new  game  are  obtained  from  the  rules  of  the  (c,  r)-game  by  reversing  the  order 
of  the  second  and  third  moves.  Thus,  the  rules  of  the  Ajtai-Fagin  (c,  7*)-game  are 
as  follows. 

1.  The  duplicator  selects  a  member  of  <S  to  be  Gq. 

2.  The  spoiler  colors  Go  with  the  c  colors. 

3.  The  duplicator  selects  a  member  of  S  to  be  Gi. 

4.  The  duplicator  colors  Gi  with  the  c  colors. 

5.  The  spoiler  and  duplicator  play  an  r-game  on  the  colored  Go,  Gi. 

The  winner  is  decided  as  before.  Thus,  in  the  Ajtai-Fagin  (c,  r)-game,  the 
spoiler  must  commit  himself  to  a  coloring  of  Go  with  the  c  colors  before  knowing 
what  Gi  is.  In  order  to  contrast  it  with  the  Ajtai-Fagin  (c,r)-game,  we  may 
sometimes  refer  to  the  (c,r)-game  as  the  original  (c,r)-game  (or  the  original 
monadic  NP  game).  In  spite  of  the  fact  that  it  seems  to  be  harder  for  the  spoiler 
to  win  the  Ajtai-Fagin  (c,r)-game  than  the  original  (c,r)-game,  we  have  the 
following  analogue  to  Theorem  2. 

Theorem  3.  [AF90]  Let  S  be  a  class  of  graphs.  S  is  in  monadic  NP  iff  there  are 
c^r  such  that  the  spoiler  has  a  winning  strategy  in  the  Ajtai-Fagin  (c,7*)-^ame 
overS. 

The  next  theorem  is  an  immediate  consequence  of  Theorems  2  and  3: 
Theorem  4.  Let  S  be  a  class  of  graphs.  The  following  are  equivalent. 

1.  For  every  c,r,  the  duplicator  has  a  winning  strategy  in  the  original  (c,r)- 
gamG  over  S. 

2.  For  every  c',r',  the  duplicator  has  a  winning  strategy  in  the  Ajtai-Fagin 

r')- game  over  S . 

Theorem  4  gives  a  precise  sense  in  which  the  original  monadic  NP  game  and 
the  Ajtai-Fagin  monadic  NP  game  are  equivalent.  Later,  we  shall  see  stronger 
versions  of  this  equivalence. 
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4  Inseparability 

In  this  section,  we  introduce  a  notion  of  inseparability.  In  this  section  and  the 
next  section,  we  use  inseparability  to  give  stronger  versions  of  the  equivalence 
between  the  original  monadic  NP  game  and  the  Ajtai-Fagin  monadic  NP  game. 
This  notion  of  inseparability  allows  us  to  make  sense  of  the  notion  of  “the  graphs 
used  in  a  game” ,  so  that  we  can  consider  statements  such  as  “the  same  family 
of  graphs  used  in  the  Ajtai-Fagin  monadic  NP  game  can  be  used  in  the  original 
monadic  NP  game  to  prove  that  a  problem  is  not  in  monadic  NP.” 

We  define  a  variation  of  the  original  (c,r)-game  over  a  class  5,  by  replacing 

5  and  S  by  arbitrary  classes  Qq  and  Qi  of  graphs.  Thus,  let  Qo  and  Qi  be  classes 
of  graphs,  and  let  c,r  be  positive  integers.  We  define  the  original  {c,r)-gaTne 
over  Qo,  Qi  to  have  the  following  rules. 

1.  The  duplicator  selects  a  member  of  Qo  to  be  Go- 

2.  The  duplicator  selects  a  member  of  Qi  to  be  Gi. 

3.  The  spoiler  colors  Go  with  the  c  colors. 

4.  The  duplicator  colors  Gi  with  the  c  colors. 

5.  The  spoiler  and  duplicator  play  an  r-game. 

The  winner  is  decided  as  before. 

We  say  that  ^Oj^i  (c,r)-inseparablei  if  the  duplicator  has  a  winning 
strategy  in  the  original  (c,r)-game  over  Qo,Qi-  In  particular,  if  th^ duplicator 
has  a  winning  strategy  in  the  original  (c,  r)-game  over  S,  then  S,S  are  (c,  r)- 
inseparablei.  If  it  is  the  spoiler,  rather  than  the  duplicator,  who  has  a  winning 
strategy,  then  we  say  that  are  (c,r)-separablei. 

Similarly  to  before,  we  define  the  Ajtai-Fagin  (c,r)-pame  over  Qo,Qi  by  ex¬ 
changing  the  order  of  the  second  and  third  moves  in  the  original  game.  As  before, 
the  difference  between  the  original  (c,  r)-game  and  Ajtai-Fagin  (c,  r)-game  is  that 
in  the  Ajtai-Fagin  game,  the  spoiler  must  commit  himself  to  a  coloring  of  Go 
before  knowing  which  graph  the  duplicator  selects  as  Gi . 

We  have  the  following  strengthened  version  of  Theorem  4. 

Theorems.  LetQo,Q\  be  classes  of  graphs.  The  following  are  equivalent 

1.  For  every  c,r,  the  duplicator  has  a  winning  strategy  in  the  original  (c,  r)- 
game  over  Qo,Qi- 

2.  For  every  c\F ,  the  duplicator  has  a  winning  strategy  in  the  Ajtai-Fagin 
{d ,r^)-game  overQo,Qi. 

We  say  that  Qo,  Qi  are  (c,  r)-inseparable2  (resp.,  (c,  r)-separable2)  if  the  dupli¬ 
cator  (resp.,  spoiler)  has  a  winning  strategy  in  the  Ajtai-Fagin  (c,r)-game  over 
QoiQi- 

The  next  proposition  follows  immediately  from  the  definitions.  It  says,  in¬ 
tuitively,  that  if  the  duplicator  has  a  winning  strategy  in  the  original  monadic 
NP  game,  then  he  has  a  winning  strategy  in  the  Ajtai-Fagin  monadic  NP  game, 
with  the  same  choices  of  graphs.  This  is  what  we  would  expect,  since  intuitively, 
it  is  even  easier  for  the  duplicator  to  win  the  Ajtai-Fagin  game  than  the  original 
game. 
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Proposition  6.  LetQo^Gi  be  classes  of  graphs.  IfQo,Gi  dre  {Cyr)-inseparablei, 
then  GoiQi  dre  (c,  r)-inseparable2 . 

As  we  shall  see  (Theorem  9),  the  converse  is  false.  We  are  interested  in  com¬ 
paring  inseparabilityi  and  inseparability2  to  compare  the  graphs  that  can  be 
used  in  a  proof  that  a  property  is  not  in  monadic  NP  using  Ajtai-Fagin  monadic 
NP  games  and  using  the  original  monadic  NP  games.  An  example  of  this  rea¬ 
soning  appears  in  Section  6. 

5  Ajtai-Fagin  games  are  no.  stronger 

The  next  theorem  is  a  strengthening  of  Theorem  5.  It  is  a  partial  converse  to 
Proposition  6.  It  tells  us  that  for  each  c,r,  there  are  such  that  if  Go,Gi  are 
(c',r ')-inseparable2,  then  Goi  Qi  are  (c,  r)-inseparablei .  In  fact,  we  can  let  r*  ~  r. 
As  we  shall  see  (Theorem  9),  we  cannot  always  let  d  =  c.  Hence,  the  converse 
of  Proposition  6  is  false,  so  we  must  settle  for  a  partial  converse. 

Theorem  7.  Let  c,  r  be  positive  integers.  There  is  d  such  that  for  every  Go ,  Qi 
that  are  {d ^T)-inseparable2,  also  Go,Oi  are  (c,r)-inseparablei. 

This  theorem  is  rather  powerful,  since  it  guarantees  the  existence  of  a  win¬ 
ning  strategy  for  the  duplicator  in  the  original  game  (with  a  certain  choice  of 
parameters)  given  only  the  existence  of  a  winning  strategy  for  the  duplicator 
in  the  Ajtai-Fagin  game  (with  another  choice  of  parameters).  Intuitively,  for  a 
given  choice  of  c,  r,  we  use  bigger  graphs  in  the  original  (c,  r)-game  than  in  the 
Ajtai-Fagin  (c,r)-game,  since  in  the  original  game  we  use  Go,  Q\  that  correspond 
to  the  Ajtai-Fagin  game  with  more  colors.  Note  that  the  choice  of  d  is  uniform, 
over  all  possible  choices  of  Go,Q\-  As  we  shall  see  by  example  in  the  next  section. 
Theorem  7  tells  us  that  the  same  families  of  graphs  can  be  used  in  the  original 
game  as  in  the  Ajtai-Fagin  game  (such  as  to  prove  that  a  class  is  not  in  mong-dic 
NP). 

6  Directed  reachability 

As  we  noted  earlier,  Ajtai  and  the  author  introduced  their  variation  of  monadic 
NP  games  in  order  to  prove  that  directed  reachability  is  not  in  monadic  NP.  In 
this  section,  we  discuss  this  approach,  and  in  particular  discuss  various  senses  in 
which  the  original  monadic  NP  game  is  adequate  and  is  not  adequate  to  obtain 
this  result. 

Let  c  and  r  be  given.  Ajtai  and  the  author  constructed  (by  probabilistic 
methods)  a  finite  directed  graph  Go  with  points  s,t  where  there  is  a  directed 
path  from  s  to  t  in  Gq.  In  fact,  Go  consists  of  a  path  from  s  to  f  (these  edges 
in  the  path  are  called  “forward  edges”),  along  with  certain  backedges.  Thus, 
Go  is  (5,t)-connected.  Denote  the  graph  that  is  obtained  by  deleting  the  edge 
e  from  Go  by  Go  •—  e.  In  particular,  if  e  is  a  forward  edge,  then  Go  —  e  is  not 
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(s,t)-connected.  Ajtai  and  the  author  showed  that  however  the  spoiler  colors  Go 
with  the  c  colors,  there  is  a  forward  edge  e  of  Go  such  that  when  Gi  =  Go  —  e  is 
colored  in  precisely  the  same  way,  vertex  for  vertex,  as  Go  was  colored,  then  the 
duplicator  has  a  winning  strategy  in  the  r-game  played  on  Go  and  Gi  (where, 
as  before,  the  isomorphism  must  also  respect  color).  Since  Go  is  (s,  t)-connected 
and  Gi  is  not,  it  follows  from  Theorem  3  that  directed  reachability  is  not  in 
monadic  NP. 

Note  that  the  duplicator  does  not  commit  himself  to  a  choice  of  Gi  until 
the  spoiler  has  committed  himself  to  a  coloring  of  Go-  This  is  the  power  of 
Ajtai-Fagin  monadic  NP  games. 

It  is  interesting  to  see  what  would  happen  if  we  were  to  try  to  use  these  pairs 
Go,Gi  in  the  original  monadic  NP  game  rather  than  the  Ajtai-Fagin  monadic 
NP  game.  Intuitively,  in  the  original  game,  the  spoiler  knows  what  Go  and 
Gi  are  before  he  colors  Go-  This  would  be  disastrous  for  a  duplicator  whose 
coloring  strategy  is  to  color  Gi  by  simply  duplicating  the  coloring  for  Gq:  if  the 
spoiler  knew  which  edge  e  were  deleted  from  Go  to  form  Gi  =  G  -  e,  this  might 
dramatically  influence  his  coloring  of  Go  (for  example,  the  spoiler  might  color 
the  endpoints  of  e  with  special  colors).  In  the  Ajtai-Fagin  monadic  NP  game, 
the  spoiler  must  commit  himself  to  a  coloring  of  Go  before  he  knows  which  edge 
e  is  deleted.  This  makes  it  easier  for  the  duplicator  to  win. 

Ajtai  and  the  author  commented  that  they  do  not  know  how  to  prove  their 
main  result  (that  directed  reachability  is  not  in  monadic  NP)  by  using  the  origi¬ 
nal  game.J[n  ^h  a  proof,  it  would  be  necessary, ^ven  c,  r,  to  show  the  existence 
of  a  pair  Go,Gi  of  finite  directed  graphs  where  Go  is  (s,f)-connected,  Gi  is  not 
(s,  t)-connected,  and  the  duplicator  has  a  winning  strategy  in  the  (c,r)-game 
over  Go,Gi.  Since  directed  reachability  is  not  in  monadic  NP  (as  Ajtai  and  the 
author  showed),  it  follows  from  Theorem  1  that  for  each  pair  c,  r,  there  is  such  a 
pair  GojGi.  Ajtai  and  the  author  instead  used  Ajtai-Fagin  monadic  NP  games, 
and  worked  with  pairs  Go,Gi  where  Go  consists  of  a  path  from  s  to  t,  along 
with  certain  backedges,  and  where  Gi  is  the  result  of  deleting  some  forward  edge 
from  Gq.  Ajtai^n(^he  author  said  that  it  is  not  clear  that  such  a  pair  Go,Gi 
could  serve  as  Go,Gi.  The  next  theorem,  which  follows  easily  from  Th^re^T, 
resolves  this  question  by  saying  that  such  a  pair  Go,  Gi  could  serve  as  Go,Gi. 

Theorems.  For  every  c^r,  there  is  a  graph  Go  that  consists  of  a  path  from  s 
to  t,  along  with  certain  backedges,  and  a  graph  Gi  that  is  the  result  of  deleting 
some  forward  edge  from  Go,  such  that  the  duplicator  has  a  winning  strategy  in 
the  {c,r)-game  over  Go,Gi. 

It  is  important  to  note  that  in  spite  of  Theorem  8,  we  do  not  know  a  direct 
proof,  using  only  the  original  monadic  NP  game,  that  directed  reachability  is 
not  in  monadic  NP, 

Remark.  We  seem  to  need  to  use  Theorem  7,  rather  than  the  slightly  weaker 
Theorem  5,  to  prove  Theorem  8.  What  could  we  obtain  by  using  only  Theo¬ 
rem  5?  Let  Qo  consist  of  all  graphs  that  are  a  path  from  s  to  t,  along  with 
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some  backedges,  and  let  be  the  collection  of  all  graphs  that  are  the  result  of 
deleting  an  arbitrary  forward  edge  from  an  arbitrary  member  of  ^o*  We  know 
from  [AF90]  that  for  every  c^r',  the  duplicator  has  a  winning  strategy  in  the 
Ajtai-Fagin  (c',/)-game  over  QojQi.  It  follows  from  Theorem  5  that  for  every 
c,  7*,  the  duplicator  has  a  winning  strategy  in  the  original  (c,  r)-game  over  Qo,Qi. 
Therefore,  for  every  c,  r,  there  are  Gq  €  Qo  and  Gi  €  Qi  such  that  the  duplicator 
has  a  winning  strategy  in  the  (c,r)-game  over  Gq,Gi.  But  this  result  is  not  as 
strong  as  Theorem  8,  since  Gi  might  be  the  result  of  deleting  some  forward  edge 
from  some  member  of  Go  other  than  Gq. 


7  Ajtai-Fagin  games  are  stronger 

The  next  theorem  says  that  inseparability i  and  inseparability2  are  different. 
Thus,  the  converse  of  Proposition  6  is  false.  This  tells  us  that  there  are  situations 
where  the  spoiler  requires  strictly  more  colors  to  win  the  Ajtai-Fagin  game  than 
the  original  game. 

Theorem  9.  There  are  classes  Go  and  Gi  of  graphs,  and  constants  c,  r,  such 
that  Go  and  Gi  ore  {c,r)-separahlei,  hut  Go  and  Gi  are  {c,r)-inseparable2. 

The  proof  of  Theorem  9  shows  that  there  are  classes  Go » Gi  of  graphs  that 
are  (2, 2)-separablei  but  (2, 2)-inseparable2.  It  turns  out  that  Go^Gi  are  (3,2)- 
separable2.  So  in  this  example,  3  colors  are  required  for  the  spoiler  to  have  a 
winning  strategy  in  the  Ajtai-Fagin  game,  but  only  2  colors  to  have  a  winning 
strategy  in  the  original  game.  Hence,  in  this  case,  the  Ajtai-Fagin  game  is  harder 
for  the  spoiler  to  win  (and  therefore  easier  for  the  duplicator  to  win)  than  the 
original  game. 


8  How  many  more  colors  are  required? 

The  contrapositive  of  Theorem  7  says  that  for  each  choice  of  the  number  c  of 
colors  and  the  number  r  of  rounds,  there  is  c'  such  that  whenever  Go^Gi  are 
(c,  r)-separablei ,  then  Go)Gi  are  (c',r)-separable2.  Let  us  denote  the  minimal 
such  value  of  c'  by  F{c,r).  Intuitively,  when  the  spoiler  can  win  the  original 
game  with  c  colors  and  r  rounds,  then  c'  =  F(c,r)  is  the  number  of  colors  the 
spoiler  needs  to  win  the  Ajtai-Fagin  game.  We  saw  in  Theorem  9  that  there  are 
c,  r  where  F{c,  r)  >  c.  This  says  that  the  spoiler  requires  strictly  more  colors  to 
win  the  Ajtai-Fagin  game  than  the  original  game.  How  many  more  colors  are 
required?  That  is,  how  much  bigger  does  F{c,  r)  need  to  be  than  c?  We  now 
give  an  upper  bound  on  F(c,r).  Define  /  by  letting  /(O)  =  2r^  -|-rlog2C,  and 
f(m  -hi)  =  for  each  m. 


Theorem  10.  F(c,r)  <  cf{r  -h  2). 
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Note  that  the  upper  bound  cf{r  +  2)  in  Theorem  10  contains  a  tower  of  r  +  2 
exponents,  where  the  top  exponent  is  a  polynomial  in  t  and  log2  c.  This  repre¬ 
sents  a  nonelementary  growth  rate.  We  conjecture  that  there  is  a  corresponding 
nonelement  ary  lower  bound. 

Acknowledgments:  The  author  is  grateful  to  Phokion  Kolaitis  and  Moshe 
Vardi  for  helpful  discussions. 
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Abstract.  We  give  an  AC®  upper  bound  on  the  complexity  of  first-oder 
queries  over  (infinite)  databases  defined  by  restricted  linear  constraints. 
This  result  enables  us  to  deduce  the  non-expressibility  of  various  usual 
queries,  such  as  the  parity  of  the  cardinality  of  a  set  or  the  connectivity 
of  a  graph  in  first-order  logic  with  linear  constraints. 


1  Introduction 

Since  its  inception  in  the  early  70’s,  Codd’s  relational  model  of  data  [Cod70]  has 
been  the  standard  framework  of  much  work  on  relational  databases  and  query 
languages.  The  almost  contemporary  renewal  of  “finite  model  theory”  (which 
dates  back  to  the  Ph.D.  dissertation  of  Ron  Fagin  in  1973)  has  offered  a  logical 
counterpart  to  this  development.  So  far,  Finite  Model  Theory  has  been  chiefly 
concerned  with  the  study  of  extensions  of  first-order  theory  and  has  greatly 
contributed  to  a  better  understanding  of  the  expressibility  and  the  complexity 
of  relational  query  languages.  In  short,  Finite  Model  Theory  and  Codd’s  rela¬ 
tional  model  have  proven  to  be  quite  appropriate  to  the  study  and  design  of 
languages  for  systems  manipulating  finite  relational  data.  But,  since  they  com¬ 
pel  all  relations  to  be  effectively  represented,  they  are  no  longer  adequate  to  new 
applications  in  databases,  such  as  spatial  (geographic)  or  temporal  databases, 
which  obviously  require  the  use  of  infinite  sets.  Of  course,  it  is  unreasonable,  from 
a  mere  computational  point  of  view,  to  jump  directly  from  the  class  of  all  finite 
structures  to  the  class  of  all  countable  structures.  For  example,  a  straightforward 
extension  of  the  relational  model  would  require  infinite  representation  (s)  of  in¬ 
finite  data.  One  must  consider  more  subtle  (and  more  efficient)  generalizations, 
where  the  data  are  handled  by  “finite  means” . 
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Such  generalizations  have  been  the  subject  of  various  attempts  in  recent 
years;  the  most  promising  ones  draw  their  inspiration  from  already  established 
research  areas  either  in  logic  (the  study  of  recursive  structures  in  classical  model 
theory  and  effective  algebra)  or  in  computer  science  (the  constraint  programming 
paradigm). 

Recursive  structures  (i.e.  relational  structures  over  a  countable  domain,  say 
the  set  of  natural  numbers,  where  every  relation  is  a  recursive  set  of  tuples) 
have  been  presented  by  Hirst  and  Harel  [HH93]  as  a  good  alternative  to  finite 
structures.  They  have  come  up  with  an  important  trade-off  between  the  class  of 
structures  taken  as  semantics  and  the  class  of  admissible  queries,  which  poses 
the  challenging  problem  of  exhibiting  interesting  classes  that  lie  between  the 
recursive  and  the  highly  symmetric  ones. 

The  constraint  database  models  introduced  by  Kanellakis,  Kuper  and  Revesz 
in  their  seminal  paper  [KKR90]  and  convincingly  advocated  in  [KG94],  is  an¬ 
other  powerful  generalization  of  Codd’s  relational  model.  In  this  new  paradigm, 
instead  of  tuples,  queries  act  on  “generalized  tuples”  expressed  as  quantifier-free 
first-order  constraints  in  a  decidable  theory  adequate  to  definite  purposes.  A  gen¬ 
eralized  (or  finitely  representable  in  our  terminology)  relation  is  a  conjunction 
of  such  constraints,  interpreted  in  the  domain  of  a  given  model  of  the  decidable 
theory.  Interesting  (and  hopefully  powerful  enough)  constraint  query  languages 
are  therefore  obtained  by  coupling  the  relational  calculus  or  some  version  of 
Datalog  with  the  theory  of  dense  linear  orders  or  the  theory  of  real  closed  fields. 

The  expressive  power  and  the  complexity  of  first-order  logic  over  finitely 
representable  databases  is  still  far  from  being  clearly  understood.  Nonethe¬ 
less,  a  series  of  complexity  and/or  expressibility  bounds  have  been  exhibited  in 
[KKR90,  KG94,  GS94,  GS95].  In  particular,  Kanellakis  and  Goldin  have  thor¬ 
oughly  investigated  the  class  of  constraints  expressed  in  C  =  {=,<}  over  a 
dense  order  and  shown  that  every  first-order  query  (in  C)  over  such  constraint 
databases  can  be  computed  in  constant  parallel  time  (uniform  AC°)  with  re¬ 
spect  to  the  size  of  the  database.  The  latter  result,  combined  with  lower  bounds 
on  the  complexity  of  queries  like  Parity  and  Connectivity  immediately  yields 
non-expressibility  corollaries  [GS94].  It  seems  highly  probable  that  similar  non- 
expressibility  results  still  hold  when  the  language  of  constraints  is  enriched  with 
addition  and  even  multiplication.  In  the  present  paper,  we  aim  to  make  one  step 
forward  in  this  direction  by  considering  linear  constraint  (expressed  in  {=,  <,  +}) 
instead  of  dense-order  ones.  We  shall  not  be  able  to  produce  a  similar  complex¬ 
ity  upper  bound  for  the  full  case  (linear  first-order  queries  over  linear  constraint 
databases).  Fortunately  enough,  we  exhibit  a  restricted  class  of  linear  constraint 
databases  to  which  Kanellakis  and  Goldin’s  AC®  upper  bound  can  be  extended. 
The  main  results  can  be  summed  up  as  follows  (Z  is  the  set  of  integers  and  Q 
the  set  of  rationals,  other  notions  will  be  defined  in  the  following  sections): 


Theorem  5.2  Every  first-order  query  in  {=,<,+}UQ  over  structures  finitely 
representable  in  {=,^,4-}  U  Z  with  the  number  of  occurrences  of  -I-  in  every 
constraint  uniformly  bounded,  can  be  evaluated  in  AC®. 
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The  previous  theorem  is  proved  assuming  a  binary  encoding  of  the  integers. 
It  does  not  carry  over  in  the  general  case  with  no  uniform  bound  on  the  number 
of  occurrences  of  -h  in  every  constraint  in  the  inputs.  We  can  therefore  conclude 
that  the  data  complexity  of  first-order  queries  over  linear  constraint  databases 
is  not  in  AC°  in  general.  Kanellakis  and  Goldin  [KG95]  suggested  to  study 
the  data  complexity  of  first-order  queries  over  linear  constraint  databases  in 
the  case  where  integers  are  encoded  in  unary.  We  prove  that  under  the  latter 
encoding  assumption,  the  AC°  upper-bound  holds  in  the  general  case.  We  think 
that  the  theorem  proven  here  constitutes  a  significant  improvement  since  linear 
constraints  are  far  more  expressive  than  the  dense-order  ones.  As  a  consequence, 
we  get  the  following  corollary. 

Theorem  6.1  Parity,  graph  connectivity,  and  region  connectivity  are  not  first- 
order  definable  with  linear  constraints. 

Note  that  the  first-order  undefinability  of  parity  and  graph  connectivity  has 
been  obtained  independently  by  Paredaens,  Van  den  Bussche  and  Van  Gucht 
[PVV95].  The  main  theorem  (Theorem  5.2)  does  not  carry  over  in  presence 
of  multiplication.  Nevertheless,  we  conjecture  that  its  corollary  (Theorem  6.1) 
holds  for  polynomial  constraints.  Proofs  in  this  paper  are  made  in  the  case  of 
the  rational  numbers.  The  undefinability  results  carry  over  in  the  case  of  linear 
constraints  over  other  domains  such  as  the  natural  numbers,  the  integers,  or  the 
reals  for  instance. 

The  paper  is  structured  as  follows.  In  Section  2,  we  review  and  discuss  some 
results  aiming  at  initiating  an  elementary  model  theory  for  different  classes  of 
countable  structures.  Section  3  is  devoted  to  basic  definitions  and  examples  of 
finitely  representable  databases.  Section  4  exhibits  an  algebraic  language  that  is 
a  procedural  equivalent  of  first-order  logic  over  finitely  representable  databases. 
The  algebra  is  used  in  Section  5  to  prove  the  main  theorem,  from  which  we 
infer  the  non-expressibility  results  of  Section  6.  Throughout  the  paper,  we  as¬ 
sume  familiarity  with  complexity  classes  defined  by  families  of  boolean  circuits, 
especially  NC  (functions  computable  in  polylogarithmic  time  with  a  polynomial 
amount  of  hardware)  and  AC°  (functions  computable  in  constant  time  with  a 
polynomial  amount  of  hardware).  For  more  details  on  complexity  classes,  we 
refer  to  [Joh90]. 

2  Restricted  Classes  of  Models 

In  this  section,  we  emphasize  some  logical  consequences  of  the  decision  to  work 
with  subclasses  of  countable  models.  In  particular,  we  investigate  conditions  un¬ 
der  which  the  compactness  or  the  completeness  theorem  do  not  hold.  It  has 
been  known  for  long  that  restricting  oneself  to  finite  structures  ruins  compact¬ 
ness  and  completeness.  On  the  contrary,  extending  the  semantics  to  all  countable 
structures  ensures  compactness  (a  direct  consequence  of  the  Lowenheim-Skolem 
Theorem).  In  this  section,  we  fix  a  purely  relational  signature  <j  —  {i?i, . . . ,  jR„} 
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(sometimes,  one  needs  that  at  least  one  of  the  iJi’s  is  of  arity  ^  2).  All  struc¬ 
tures  will  be  of  the  form  *4  =  (A,  ,  i^„},  with  A  some  countable  set  (say  a 

subset  of  natural  numbers).  E  A  is  finite,  one  recovers  the  usual  notion  of  a  finite 
structure.  If  A  and  all  Ri^s  are  recursively  enumerable  (respectively  recursive, 
primitive  recursive),  then  A  is  said  to  be  recursively  enumerable  (respectively 
recursive,  primitive  recursive).  Let  Strfin  (respectively  Strr.e.,  Strreo  Strp,r.) 
denote  the  set  of  all  finite  (respectively  recursively  enumerable,  recursive,  prim¬ 
itive  recursive)  structures,  and  Vfin  (respectively  Vr.e.,  Vrec,  Ip.r.)  denote  the 
set  of  all  cr-sentences  true  in  all  structures  of  Sir  fin  (respectively  StVr.e.-,  Strrec-, 
Strp,r.).  The  following  theorem,  due  to  Mostowski  [Mos57]  and  Vaught  [Vau60], 
establishes  that,  for  any  reasonable  class  of  “constructive  structures”,  the  com¬ 
pleteness  theorem  fails: 

Theorem  2.1  (Mostowski  [Mos57]  and  Vaught  [Vau60])  Let  V  be  a  set  of  cr- 
sentences.  If  Vr,e.  QVC  Vfim  then  V  is  not  recursively  enumerable.  Moreover, 
if  Vr.e.  Qy  Q  Vp.r.i  then  V  is  not  arithmetical. 

Let  us  now  consider  with  more  details  the  class  For  a  =  {£?},  where 

E  is  binary,  it  has  already  been  the  focus  of  some  attention  in  the  past,  par¬ 
ticularly  from  combinatorists.  Indeed,  it  has  been  proved  that  switching  from 
finite  graphs  to  recursive  ones  can  tremendously  increase  the  data  complexity 
of  usual  problems.  For  instance,  the  existence  of  a  Euler  path  (which  can  be 
decided  in  polynomial  time  in  the  finite  case)  becomes  ilg-complete,  thus  un- 
decidable  [Bea76],  while  Hamiltonicity  (a  well-known  NP-complete  problem  for 
finite  graphs)  becomes  -complete,  thus  even  not  in  the  arithmetical  hierarchy 
[Har91]. 

More  recently,  Hirst  and  Harel  [HH93]  studied  the  recursive  structures  from 
a  database  point  of  view.  Some  of  their  results  are  worth  mentioning.  It  is 
known  that  very  primitive  relational  operators,  e.g.  projections,  do  not  pre¬ 
serve  the  recursiveness  of  relations:  if  T{x,  y,  z)  C  is  the  primitive  recursive 
relation  expressing  that  “the  Turing  machine  halts  on  input  z  in  x  steps” , 
then  3xT{x,y,z)  C  expresses  the  halting  problem.  As  a  consequence,  if 
one  wants  queries  to  be  computable,  even  the  relational  calculus  (i.e.  first-order 
logic)  is  too  expressive  a  language.  Hirst  and  Harel  show  that,  over  the  class 
of  all  recursive  countable  databases,  quantifier-free  first-order  logic  is  complete 
with  respect  to  the  class  of  computable  and  generic  (a  consistency  criterion  ex¬ 
pressing  commutation  with  isomorphisms)  queries.  Consequently,  they  define  a 
drastically  restricted  subclass  of  recursive  databases,  called  “highly  symmetric”, 
whose  behavior  with  respect  to  completeness  (a  version  of  Chandra  and  HarePs 
QL  [CH80])  and  BP-completeness  [Ban78,  Par78]  (first-order  logic)  resemble  the 
class  of  finite  databases.  Thus,  they  have  come  up  with  an  important  trade-off 
between  the  class  of  structures  taken  as  semantics  and  the  class  of  admissible 
queries,  which  poses  the  challenging  problem  of  exhibiting  interesting  classes 
that  lie  between  the  recursive  and  the  highly  symmetric  ones.  Seemingly,  the 
constraint  database  model  offers  a  framework  for  the  definition  of  such  classes. 

In  their  notes  on  recursive  model  theory  [HH94],  Hirst  and  Harel  prove  that 
the  compactness  theorem  fails  for  the  class  of  all  countable  recursive  structures. 
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Their  argument  does  not  lend  itself  naturally  to  arbitrary  subclasses  of  countable 
structures.  J.  Vaananen  [Va94]  suggested  that  the  compactness  theorem  should 
fail  for  any  subclass  of  countable  structures  containing  all  finite  structures  and 
no  infinite  countable  structure  elementary  equivalent  to  a  fixed  (infinite)  locally 
finite  structure  (a  structure  is  locally  finite  if  every  sentence  of  its  theory  has  a 
finite  model). 

3  Linear  Constraint  Databases 

Constraint  databases  may  be  defined  over  various  sorts  of  constraints,  such  as 
dense-order  constraints,  polynomial  constraints  over  the  reals,  etc.  Here  we  in¬ 
troduce  a  general  paradigm  independent  of  the  choice  of  the  constraints.  Let  C 
be  a  first-order  language  with  equality  and  D  some  non  empty  set.  We  consider 
an  ^-structure,  D,  with  universe  J9.  X>  is  called  the  domain- structure.  Finally, 
let  T  be  the  first-order  theory  of  V. 

Consider  for  instance,  C  —  {^,  +}  U  Q.  The  structure  we  shall  be  concerned 
with  in  the  present  paper  is  D  =  (Q,  (9)g€Q),  the  structure  of  the  linearly 

ordered  set  of  the  rational  numbers  with  addition  and  all  rational  constants,  and 
T  is  the  theory  of  dense  orders  without  endpoints  and  with  addition.  [Another 
traditional  example  is  £  =  {^,+,  x,0, 1},  V  =  (M,  -H,  x,0, 1)  (the  field  of 

reals)  and  T  is  the  theory  of  the  ordered  real  closed  fields.] 

Let  a  =  {ill,  •••j'Rn}  be  a  signature  (or  a  database  schema)  such  that  C  n 
cr  =  0,  where  ili,...,iln  are  relation  symbols.  We  distinguish  between  logical 
predicates  (e.g.,  =,  ^)  in  C  and  relations  in  a.  We  next  introduce  a  restricted 
definition  of  finitely  representable  structures  [GS94].  We  consider  expansions  of 
T>  to  (X.  Intuitively,  the  relations  in  a  constitute  a  database  in  the  context  of  V. 

Definition  3.1  Let  5  C  be  some  /c-ary  relation.  The  relation  S  is  finitely 
representable  in  C  over  T)  [C-representable  for  short)  if  there  exists  a  quantifier 
free  formula  <^(a;i,  ...,0;*:)  in  £  with  k  distinct  free  variables  a;i,...,xjfc  such  that: 

Vdi, ..., E  Z),  (oi , ...,  Cl;.)  E  ^  ^(fli, ..., a/-) 

Let  be  an  expansion  of  V  to  cr.  The  structure  A  is  finitely  representable  (over 
V)  if  for  every  relation  symbol  il  in  cr,  is  £-representable  (over  V). 

Kanellakis,  Kuper,  and  Revesz  [KKR90]  introduced  the  concept  of  a  fc-ary 
generalized  tuple,  which  is  a  constraint  expressed  as  a  conjunction  of  atomic 
formulas  in  £  over  k  variables.  A  fc-ary  finitely  representable  relation  (or  gen¬ 
eralized  relation  in  [KKR90])  is  then  a  finite  set  of  fc-ary  generalized  tuples.  In 
the  remainder  of  the  paper,  we  focus  on  the  language  £  =  {^,+}UQ  and  the 
£-structure  V  =  (Q,  (g),^^?)*  Therefore,  constraints  will  be  composed  of 

linear  equations  or  inequalities  of  the  form: 

p  p 

^  ^  Oi  ajj  =  flO  ?  ^  ^  Oii  ^  Oq 

t=l  i=l 
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where  the  Xi^s  denote  variables  and  the  a^’s  are  integer  constants  (note  that 
rational  constants  can  always  be  avoided  in  linear  equations  and  inequalities). 

A  (database)  instance  (of  a)  iso.  mapping  which  associates  with  each  Ai-ary  re¬ 
lation  symbol  .R  in  cr  a  quantifier-free  formula  in  disjunctive  normal  form  (DNF) 
with  k  distinct  variables.  Clearly,  each  instance  of  a  corresponds  to  the  restric¬ 
tion  of  a  finitely  representable  structure  to  cr.  In  practice,  we  assume  that  the 
databases  contain  the  formula  defining  their  relations.  Instances  will  be  denoted 
by  /,  J,  etc. 

Note  that  the  class  K  of  (7-instances  is  effectively  enumerable  if  the  cardinality 
of  the  language  C  is  countable.  Moreover,  if  V  is  recursive,  then  instances  are 
recursive.  K  has  interesting  closure  properties.  It  is  closed  under  finite  union 
and  intersection  and  moreover  under  complementation.  This  differs  from  finite 
model  theory  (the  complement  of  a  finite  model  is  not  finite).  Our  main  goal 
is  to  investigate  the  expressive  power  of  first-order  logic  over  the  class  of  linear 
constraint  databases.  We  consider  partial  recursive  classes  of  ^-representable 
databases  and  ask  whether  they  can  be  captured  by  a  first-order  sentence  in  £. 

In  the  main  theorem,  we  restrict  our  attention  to  a  family  of  database  in¬ 
stances,  called  “fc-bounded”  instances.  Intuitively,  /c-bounded  linear  instances 
are  defined  with  equations  and  inequalities  with  bounded  variable  factors.  We 
shall  prove  that  first  order  queries  over  /c-bounded  instances  can  be  evaluated 
in  AC®  in  terms  of  the  database  size  (data  complexity).  Following  is  a  formal 
definition  of  A:-boundedness. 

Definition  3.2  Let  A;  ^  0  be  an  integer.  An  atomic  formula  is  k-bounded  if  it  is 
in  {^,  -|-}UZ  (no  rationals)  and  contains  at  most  A;  occurrences  of  the  (function) 
symbol  “-f”.  A  quantifier-free  formula  is  k-bounded  if  each  atomic  formula  in  it 
is  A;-bounded.  Finally,  an  instance  of  signature  a  is  k-bounded  if  for  each  relation 
symbol  R,  the  associated  quantifier-free  formula  is  /c-bounded.  We  denote  by 
ACfc(<7),  or  simply  /C^,  the  family  of  all  A;-bounded  instances  over  a. 

A  A;-bounded  constraint  has  the  following  form: 

p 

flj  Xi 

i=l 

where  0  is  a  predicate,  the  at’s  are  integers,  and  \ai\  -H  ao  <  A;  -I-  2  (where 
jail  denotes  the  absolute  value  of  ai,  and  ao  =  1  if  oo  0)  and  oq  =  0  otherwise). 

Note  that  when  A;  =  0,  /Co  is  exactly  the  set  of  dense  order  constraints  which 
were  studied  in  [KKR90,  KG94,  GS94].  For  this  class  of  constraints,  an  upper 
bound  on  the  complexity  of  the  first-order  queries  expressed  in  the  language 
{<}  U.Q  is  known: 

Theorem  3.1  [KG94]  The  data  complexity  of  first  order  logic  in  the  language 
{<}  U  Q  over  the  family  /Co  of  dense  order  instances  is  in  AC®. 

The  proof  of  this  result  is  based  on  a  canonical  encoding  of  dense  order  in¬ 
stances  into  finite  instances.  This  is  possible  since  dense  order  instances  admit 


^  0  flo 
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very  simple  geometrical  decompositions  in  terms  of  atomic  “cells”  [Col75]  of  sim¬ 
ple  shapes.  Note  that  the  encoding  itself  is  not  in  AC®.  A  specific  algebra  working 
on  finite  structures  is  introduced  in  [KG94],  which  simulates  the  manipulation 
of  dense  order  instances. 

4  First-order  Query  Languages 

We  define  FO^  as  first-order  logic  with  linear  constraints,  i.e.  over  the  language 
£  =  4-}uQ.  We  introduce  in  this  section  an  algebra  ALG£  for  finitely  repre¬ 

sentable  databases,  and  prove  its  equivalence  with  FO^.  This  algebra  is  similar 
to  Codd’s  algebra  for  finite  relations  [Cod70],  but  the  operators  apply  to  finite 
representations  of  possibly  infinite  sets.  The  algebra  consists  of  the  following 
operations:  cartesian  product,  x,  selections  (or=,  a<,  and  (t^),  projection,  tt,  set 
operations  (union,  U,  intersection,  n,  and  set  difference,  -),  and  rename,  p. 

The  algebra  operations  are  performed  on  sets  of  generalized  tuples,  i.e.  on 
quantifier-free  formulas  in  DNF.  But  unlike  Kanellakis  and  Goldin  [KG94],  we 
do  not  assume  special  encoding  for  relations  and  generalized  tuples.  On  the  other 
hand,  our  algebra  can  also  be  viewed  as  a  simplified  sublanguage  of  the  algebra 
of  Paredaens,  Van  den  Bussche  and  Van  Gucht  [PVV94]  (which  also  includes 
multiplication). 

The  algebra  will  serve  as  a  mere  technical  tool  for  the  proof  of  the  main 
theorem.  We  should  note  that  it  has  no  important  preservation  property  with 
respect  to  the  size  of  (the  representation  of)  a  database  or  /c-boundedness.  How¬ 
ever,  such  properties  are  not  necessary  for  our  purpose.  We  shall  instead  use 
upper  bounds  on  the  parameters  (size  and  degree  of  boundedness)  of  a  database 
generated  by  the  application  of  an  operation  of  the  algebra  (see  Section  5  for  an 
in-depth  study). 

We  now  define  the  algebra  operators.  Suppose  is  an  n-ary  relation  repre¬ 
sented  by  a  quantifier-free  formula,  y),  of  the  form: 

»=i  j=i 

where  the  are  atomic  formulas.  Then,  we  also  denote  the  representation  (p 
as  a  collection  of  generalized  tuples  U  in  the  set  notation: 

t 

1  ^  i  ^  kj  ti  =  (pij 

j=i 

Furthermore,  if  I  is  an  instance  over  signature  a  and  R  e  we  consider  the 
relation  /(i?)  as  a  set  of  generalized  tuples  as  above.  We  also  assume  that  at¬ 
tributes  (columns)  of  relations  have  names  and  for  each  attribute  name  A,  there 
is  a  distinct  variable  xa  associated  with  it.  Attribute  names  are  usually  denoted 
by  A,  C, . . .  (and  possibly  with  subscripts).  When  the  context  is  clear,  we  may 
blur  the  distinction  between  variables  and  attribute  names. 
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Definition  4.1  Let  cr  be  a  signature.  The  family  of  algebraic  expressions  (over 
a)  is  defined  inductively  as  follows: 

1.  (R)  and  {A  :  Q)  are  atomic  expressions,  where  G  cr  is  a  relation  symbol, 
and  A  is  an  attribute  name.  The  set  of  attributes  is  the  set  of  attributes  of 
J?  or  {A},  respectively. 

Suppose  now  that  ei  and  62  are  two  algebraic  expressions. 

2.  (Cartesian  product)  If  ei  and  62  have  disjoint  sets  of  attribute  names,  then 
(ei  X  62)  is  also  an  expression. 

3.  (Selection)  If  F  is  a  selection  formula  (defined  below)  involving  only  attribute 
names  of  ei,  then  ((Tj?  ei)  is  an  expression.  A  selection  formula  is  an  atomic 
formula  of  one  of  the  following  three  forms: 

*1=^2,  ti-\-t2~h, 

where  are  attribute  names  or  constants  (in  Q). 

4.  (Projection)  If  ei  has  attributes  {Ai , . . . ,  An},  and  moreover  {Bi , . . . ,  }  C 

{Ai, . . . ,  An},  then  (7rBi,...,Bfc  ei)  [or  (TT^ei)  if  A;  =  0]  is  an  expression. 

5.  (Set  operations)  If  61,62  have  exactly  the  same  set  of  attributes,  then  (ci  — 
62)5  (ei  n  62),  and  (61  U  62)  are  expressions. 

6.  (Rename)  If  A,  B  are  two  attribute  names  and  A  is  an  attribute  of  61  but  B 
is  not,  then  (pa-j-b  ^i)  is  also  an  expression. 

We  now  describe  the  semantics  of  the  algebra.  (Note  that  the  operators 
work  directly  on  generalized  tuples,  so  the  semantics  is  given  with  respect  to 
generalized  tuples.)  Suppose  that  I  is  an  instance  of  <7,  and  e  is  an  expression 
over  <7.  The  result  of  e  on  /,  denoted  by  e{I),  is  defined  inductively  as  follows: 

1.  (a)  If  6  =  (B),  e{I)  =  I{R)  (a  set  of  generalized  tuples). 

(b)  If  6  =  (A  :  Q),  e(/)  =  {xa  =  x a} ^  where  xa  is  the  variable  corresponding 
to  the  attribute  A. 

2.  If  6  =  (61  X  62),  then  6(7)  =  {ti  At2\ti  e  6i(J),i2  €  62(7)}. 

3.  If  6  =  {ctf  Ci),  6(7)  ~  {tAF  \  t  e  Cl (7)},  where  each  attribute  name  A  in  F 
is  replaced  with  the  corresponding  variable  xa- 

4.  If  6  =  ^1?  is  obtained  from  ei(7)  by  “eliminating”  the 

variables  which  do  not  correspond  to  attributes  Bi  through  Bjfc.  One  proceeds 
as  follows.  Suppose  61  (7)  =  {ti, . .  -  ,tm}  and  has  attributes  Ai, . . . ,  A„  and 
{C'i,...,Cn-jb}  =  {Ai,..., A„}  -  {Bi,...,Bfc}.  We  apply  the  well-known 
Fburier-Motzkin  Elimination  method  [Sch86]  (see  below)  to  eliminate  one 
by  one  all  existentially  quantified  variables  xcir  "  i^Cn-h  i^ 
formulas  3xci  •  Each  tuple  U  then  results  in  t[.  Finally,  6(7)  = 

Wj  •  •  •  J^m}- 

5.  (a)  If  6  =  (61  U  62),  then  6(7)  =  61  (7)  U  62(7). 

(b)  If  6  =  (ei  n  62),  then  6(7)  =  {ti  A  ^2  I  ^  ei(7),t2  €  62(7)}. 
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(c)  If  e  =  (61—62),  then  e(/)  =  {fi  Ai2  |  €  6i(/),t2  €  {^2{I)y}i  where 

is  the  complement  of  R  obtained  as  follows.  Suppose  i?  =  {ii, . . . ,  is 
a  set  of  generalized  tuples  and  for  each  iyti  =  (fij.  Then  R^  is  the 
formula®  in  DNF  which  is  equivalent  to  -xpij. 

6.  If  6  =  pA~^B  ei,  then  e(/)  =  61  {I)[xa/xb]  (all  occurrences  of  xa  are  replaced 
by  xb). 

The  Fourier-Motzkin  elimination  method  (see  for  instance  [Sch86],  pp.  155- 
157)  works  as  follows.  Consider  a  generalized  tuple  t  which  defines  a  polyhedron 
Q  described  by  the  inequalities  (once  the  coefficients  of  y  have 

been  normalized) : 

a^x  +  2/  ^  Oq  for  £  =  1, . . . ,  L 
^  5%“2/^6q  for /c  =  1, . . .  j 
&x  ^  Cq  for  i  =  1, . . . ,  / 

where  x  G  ,  2/  G  Q.  One  can  show  that  after  the  “elimination”  of  y  (i.e.  aftfer 
P  has  been  projected  on  its  first  n  coordinates),  the  relation  over  x  is  exactly: 

{  X  G  I  b^x  -Bq  ^Qq-  a^x  for  all  i  and  /c,  c*x  <  Cq  for  all  f }  . 
Therefore: 


^  b^x -Bq  ^aQ-~a^xc^x  ^cl. 

It  is  easy  to  verify  that  the  algebra,  denoted  by  ALG£,  is  equivalent  to  first- 
order  logic  over  the  class  of  structures  we  consider.  The  proof  (which  is  omitted) 
is  quite  similar  to  that  of  the  equivalence  of  the  classical  relational  algebra  and 
calculus  over  finite  structures  (see  [AHV94]). 

Theorem  4.1  FO£  =  ALG£. 

We  illustrate  the  above  result  with  the  following  example. 

Example  4.1  Consider  the  following  query  over  a  binary  relation  R  with  at¬ 
tributes  A,  B: 

{z  I  3a:3y  {R(x,  j/)  A  3/  =  2x  +  «)}  . 

The  equivalent  algebra  query  is: 


TrA2<rB=Ai+A2<rAi=A+A  {R  X  (>ll  =  Q)  X  (^2  :  Q))  .  ■ 

Note  that  the  formula  may  a  priori  have  exponential  length  in  the  size  of  the  original 
formula  /)^  .  We  prove  in  the  next  section  that  it  can  be  done  in  polynomial 

length  for  the  families  of  databases  considered  here. 
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Remark.  The  combination  of  selections  and  cartesian  products  can  yield  com¬ 
plicated  forms  of  selections.  For  instance,  aB=kA+c{e)  (e  is  an  expression  with 
attributes  A^B)  can  be  expressed  as: 

TTA.B  crB=Ak-i+c(^Ak-i=Ak-2+A  ‘  *  *  ^Ai=A+A  (e  X  (Ai  :  Q)  X  •  •  ■  X  {Ak-l  :  Q)) . 

Finally,  we  discuss  the  complement  operation  used  in  defining  the  seman¬ 
tics  for  the  set  difference  operation.  Computing  the  complement  of  a  relation 
R  is  generally  a  costly  operation.  The  naive  approach  to  converting  a  formula 
/AW  of  size  n  into  DNF  might  generate  a  formula  whose  length  is  expo¬ 
nential  in  n.  However,  there  are  special  cases  where  efficient  algorithms  exist. 

Example  4.2  Suppose  R  is  a  set  of  binary  generalized  tuples  consisting  of  linear 
constraints  with  a  fixed  set  of  k  distinct  (rational)  slopes  . . . ,  afc.  We  can  view 
the  constraints  in  R  as  dividing  (Q^  into  many  “cells”  and  R  as  a  collection  of 
these  cells.  Since  each  cell  is  a  convex  polygon,  every  cell  can  be  defined  using 
at  most  2k  constraints.  It  can  then  be  verified  that  there  exists  a  representation 
of  the  complement  of  i?,  where  each  tuple  involves  at  most  2k  constraints.  Let 
S  be  the  set  of  all  possible  constraints  (involving  n  variables)  in  R  or  obtained 
from  constraints  in  R  by  changing  the  logical  predicates  and  define: 

17  =  { A  •  •  •  A <2fe  I  for  each  1  <  i  <  2A;,  Ue  S}  . 

Then,  the  complement  of  R  can  be  defined  as: 

R^  =  {teU  \  \fxt{x)  -iR(^)  }  . 

Therefore,  the  complement  can  be  computed  in  polynomial  time  for  each  fixed 

k,  ■ 

5  Complexity 

In  this  section,  we  analyze  the  data  complexity  of  first-order  queries.  We  present 
two  results:  (i)  a  known  NC  bound  [KKR90]  in  the  general  case,  and  (ii)  a  new 
AC°  bound  for  a  restricted  class  of  inputs,  namely  fc-bounded  instances  for  a 
fixed  k.  The  proof  of  this  last  result  relies  on  the  algebra  introduced  in  the 
previous  section. 

The  time  (or  space)  data  complexity  of  a  query  is  the  time  (resp.  space) 
needed  in  evaluating  the  query  in  terms  of  the  “size”  of  the  representation  of 
the  input  instances.  Formally,  we  have: 

Definition  5.1  Let  R  be  a  relation,  and  (pR  its  representation  over  the  language 
{=,^,4-}  U  Z.  The  formula  (pR  is  of  size  |<^b|  ^  n  if  0b  contains  at  most  n 
disjuncts  (tuples)  and  at  most  n  distinct  constraints,  and  the  absolute  values  of 
the  integers  occurring  in  <pR  are  bounded  by  2”  (i.e.  the  absolute  values  can  be 
represented  in  binary  notation  with  n  bits). 
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It  was  shown  in  [KKR90]  that  first-order  queries  with  polynomial  constraints 
(over  the  real  numbers)  have  NC  data  complexity.  This  result  follows  from  tech¬ 
niques,  first  introduced  in  [BKR86],  showing  that  the  theory  of  real  closed  fields 
of  fixed  dimension  (number  of  variables)  can  be  decided  in  NC.  The  same  upper 
bound  of  course  holds  in  the  case  of  linear  constraints. 

Theorem  5.1  [BKR86,  KKR90]  FO/:  is  in  NC  over  the  class  of  linear  constraint 
inputs. 

We  next  present  the  main  theorem  of  this  section  which  applies  to  a  restricted 
class  of  inputs  that  is  of  practical  interest,  namely,  /j-bounded  linear  constraint 
inputs.  Recall  that  a  fc-bounded  linear  constraint  input  is  a  relation  that  is 
finitely  representable  by  a  quantifier  free  formula  in  DNF,  such  that  in  each 
atomic  formula  occurring  in  it  there  are  at  most  k  occurrences  of  the  addition 
symbol,  and  all  constants  are  integers. 

Theorem  5.2  For  each  (fixed)  integer  A;  >  0,  FO£  is  in  uniform  AC°  over  the 
class  of  A;-bounded  linear  constraint  inputs. 

First  observe  that  Theorem  5.2  doesn’t  carry  over  for  the  general  case  of  not 
A;-bounded  linear  constraint  inputs  (with  binary  encoding  of  natural  numbers). 
Consider  a  monadic  relation  containing  a  single  tuple:  R  =  {[ax  =  b  A  x  =  h^]} 
for  arbitrary  values  of  a,  b  and  b^  in  N.  The  boolean  query  ttq  {R)  ^  0  is  true  iff 
axb'  =  b.  The  size  of  relation  R  is  essentially  the  size  of  the  three  numbers  a,  6, 
and  b'.  Multiplication  of  numbers  in  binary  notation  is  not  in  AC®  [FSS84].  We 
can  therefore  conclude  that  first-order  logic  over  linear  constraint  databases  is 
not  in  AC®. 

Theorem  5.2  extends  the  now  classical  result  that  the  relational  algebra  has 
AC®  data  complexity  over  finite  structures.  Before  presenting  the  proof  of  The¬ 
orem  5.2,  we  briefly  review  the  proof  in  the  case  of  the  relational  algebra  over 
finite  structures  as  it  is  sketched  in  [AHV94].  In  the  case  of  finite  relations,  the 
circuits  are  constructed  uniformly  as  follows.  The  gates  of  the  circuit  represent 
pairs  of  the  form  where  R  is  a  relation  name  (or  any  algebraic  expression, 
such  as  R'  X  R"),  and  t  is  a  tuple  of  the  same  arity  as  R.  The  semantics  is  that 
the  value  of  a  gate  [R,  t]  is  1  iff  R{t)  holds. 

Consider  an  algebraic  query  Q.  There  is  a  gate  of  the  form  [R,  t]  for  each  R, 
either  an  input  relation  or  an  algebraic  expression  that  is  a  sub-expression  of  the 
query  Q,  and  each  tuple  t  which  has  the  proper  arity  and  is  built  with  atomic 
constants  from  the  input  relations.  That  gives  rise  to  a  polynomial  number  of 
gates. 

The  circuit  computes  the  value  of  [Q,  s],  for  each  tuple  s  of  the  corresponding 
arity,  starting  from  the  values  of  the  [R,t],  where  R  is  an  input  relation.  Most 
operations  are  very  simple  to  simulate.  For  instance,  the  value  of  [R'  x  R",  [t^t"]] 
is  1  iff  both  [R'jf]  and  [R”^t"]  have  the  value  1.  The  only  operation  that  is 
slightly  more  complex  is  the  projection,  which  requires  unbounded  fan-in  of  the 
OR  gates. 
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In  the  case  of  constraint  databases,  the  number  of  tuples  (of  atomic  values) 
is  infinite.  Instead  of  the  tuples,  the  generalized  tuples  need  to  be  encoded.  We 
next  explain  how  the  encoding  is  done  using  gates  in  a  circuit. 

Without  loss  of  generality,  we  make  a  few  assumptions  to  simplify  the  presen¬ 
tation.  Specifically,  we  assume  that  Q  is  a  first-order  boolean  query  whose  input 
consists  of  a  single  binary  relation  R.  For  each  natural  number  n,  we  exhibit  a 
boolean  circuit  Cn,  of  constant  depth  (depending  only  upon  the  query  Q  and  the 
degree  k  of  boundedness  of  the  inputs)  with  polynomially  many  gates  in  terms 
of  n.  The  circuit  (7„  has  the  property  that  for  each  /c-bounded  input  R  with  a 
representation  (j>ji  of  size  smaller  than  n,  the  circuit  starting  on  an  encod¬ 
ing  enc{(l>R)  of  (j>R,  computes  an  encoding  of  Q{R).  The  proof  easily  extends  to 
inputs  with  several  relations,  of  arbitrary  arities,  and  to  queries  with  outputs  of 
arity  ^  1.  The  circuits  then  have  many  output  gates,  giving  an  encoding  of  (a 
representation  of)  the  output. 

The  input  (under  the  previous  assumptions)  is  encoded  as  follows.  We  first 
describe  how  to  encode  with  3n®  -I-  4n^  bits  any  (quantifier  free)  formula  of 
{=,<,+}  UZ  with  two  free  variables  of  size  n.  (i)  Integers  are  encoded  in  binary 
notation  with  n  bits,  (ii)  Constraints  of  the  form  ax  -f  /?y6>7,  where  a,  /?,  and 
7  are  integers  whose  absolute  values  are  smaller  than  2”,  and  0  is  =  or  <,  are 
encoded  on  3n  +  4  bits  as  follows: 


9a\a\P  \fi\  7  |7|  , 


where  the  bit  0  =  0  (resp.  1)  if  0  is  =  (resp.  <);  the  bit  a  =  1  (resp.  0)  if  a  is  a 
positive  (resp.  negative)  integer;  \a\  is  the  binary  representation  of  the  absolute 
value  of  a  in  n  bits;  and  similarly  for  /?  and  7. 

Since  there  are  at  most  n  constraints  in  each  tuple  and  at  most  n  tuples  in 
the  binary  relation  i?,  the  whole  encoding  of  a  formula  for  R  of  size  n  requires 
a  sequence  of  n  x  n  x  (3n  -f-  4)  =  3n®  +  4n^  bits. 

During  the  computation,  the  syntactic  objects  encoded  in  the  circuits  can 
grow  in  size.  For  instance,  bigger  integers  may  result  from  adding  integers  of  size 
n.  Similarly,  constraints  over  more  than  two  variables  are  sometimes  needed,  as 
a  result  of  an  application  of  the  cartesian  product,  for  instance.  The  cartesian 
product,  along  with  other  operations,  also  trigger  an  increase  of  the  number  of 
constraints  in  each  tuple.  Therefore,  the  number  of  bits  allocated  to  the  encod¬ 
ing  of  integers,  constraints,  and  tuples  varies  at  the  different  strata  (depths)  of 
the  circuit.  The  encoding  of  bigger  integers,  constraints  over  more  variables,  and 
tuples  containing  more  constraints,  is  done  in  the  same  manner  as  above,  by 
adding  the  required  amount  of  space.  Since  each  first  order  query  can  be  eval¬ 
uated  using  a  fixed  number  of  (algebraic)  operations,  the  required  additional 
space  can  always  be  figured  out  once  a  particular  query  is  given. 

In  the  following  we  first  discuss  the  projection  and  set  difference  operations 
in  the  algebra,  prove  two  key  lemmas  concerning  the  AC®  data  complexity  bound 
of  these  two  operations,  and  then  present  the  proof  of  Theorem  5.2. 
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The  projection  operation  requires  the  computation  of  addition,  and  repeated 
addition  (bounded  multiplication).  We  first  prove  that  (i)  the  addition  of  two 
integers,  and  thus  (ii)  the  multiplication  of  an  integer  by  a  given  constant  can 
be  done  in  uniform  AC®  with  respect  to  the  size  of  the  binary  representation  of 
the  integers. 

Lemma  5.3  The  addition  of  two  binary  integers  of  size  ^  n,  UnUn-i  *  *  ‘^lao, 
and  bnbn-i  *  *  *  can  be  done  by  constant-depth  circuits  with  n  -f  1  output 
gates  and  at  most  V{n)  gates,  where  7^  is  a  polynomial. 

Proof:  Assume  that: 


O'nO^n-l  •  *  •  ^^1^0  +  bnbn-l  *  *  *  bibo  =  Cn+lCn  * '  •  CiCq 

The  boolean  circuit  is  constructed  uniformly  with  the  following  formulas: 

Co  =  -i(ao  <=>  bo) 

i=k-l  /  / j=k-l 

Ck  =  (ufc  -4^  bk)  I  I  (fljf  V  I  A  (uj  A  6t) 

i=o  \  \  j^t+l 

for  each  1  ^  A;  <  n,  and 

/  /  i=n  \ 

Cn+1  =  \)(/  I  j  {o>j  V  bj)  j  A  (fli  A  bi) 

i=0  / 

where  is  an  abbreviation  for  a  circuit  of  depth  3  using  only  A,  and  V 
nodes:  x  =  (x  Ay)  V  (-ixA-iy).  The  depth  of  the  circuit  is  7,  and  the  number 
of  nodes  is  O(n^).  ■ 


Remark.  On  the  other  hand,  addition  of  rational  numbers  (encoded  as  pairs 
of  natural  numbers)  is  not  in  AC®.  Indeed,  this  would  imply  that  multiplica- 

o  1  1  U  “1“  X 

tion  of  natural  numbers  is  also  in  AC®.  Indeed,  consider  — h  -  =  - .  As 

X  y  y  X  X 

a  consequence,  our  proof  does  not  carry  over  to  the  case  where  databases  are 
defined  with  rational  numbers  as  parameters.  This  follows  from  the  fact  that  ad¬ 
dition  of  parameters  coming  from  the  input  constraints  is  required  to  compute 
an  application  of  projection. 

We  next  prove  that  the  projection  can  be  done  in  AC®.  More  precisely,  we 
prove  that  for  each  tuple,  there  is  a  circuit  of  fixed  depth,  with  a  polynomial 
number  of  gates,  that  computes  the  projected  tuple. 

Lemma  5.4  Let  5  be  a  /c-bounded  set  of  linear  constraints  over  n  variables 
xi,...jXn  for  some  fc,  and  i  a  positive  integer  ^  n.  The  projection  il(5)  of  S  on 
variables  {xi,  ...,Xn}  —  {a:*}  is  computable  in  AC®. 
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In  Lemma  5.4,  the  set  S  denotes  a  single  fc-bounded  tuple,  i.e.  the  total 
number  of  occurrences  of  the  addition  symbol  in  each  constraint  in  S  is  bounded 
by  A;.  It  follows  easily  that  the  projection  of  an  entire  /c-bounded  relation  can  be 
done  in  AC®.  The  circuit  contains  essentially  copies  of  the  circuit  that  computes 
the  projection  individually  for  each  tuple. 

Proof  of  Lemma  5,4:  The  AC®  upper  bound  for  the  projection  of  a  tuple 
relies  on  the  following  simple  technical  claim,  which  shows  how  addition  and 
multiplication  are  used  in  the  computation  of  the  resulting  constraints  after  a 
set  of  constraints  has  been  projected  onto  some  components. 

Claim:  Let  5  be  a  fc-bounded  set  of  linear  constraints  over  n  variables  a;i , 

n 

of  the  form:  ^a£a;^6>ao,  and  let  C{S)  be  the  set  of  variable  coefficients 
i=i 

(namely,  ai  for  each  £  ^  1),  and  Co(S)  the  set  of  constant  coefficients  (namely, 
ao)?  in  the  constraints  of  S.  Let  iJ  be  the  projection  on  variables  {a;i,  ...,Xn}  — 
{a^i}  for  some  i.  Then  the  following  holds: 

-  The  variable  coefficients  of  11  (S)  are  obtained  by  additions  and  multiplica¬ 
tions  of  variable  coefficients  in  C(5),  and 

-  The  constant  coefficients  of  il  (5)  are  obtained  by  multiplications  of  a  con¬ 
stant  coefficient  in  Co{S)  with  a  variable  coefficient  in  C(5),  and  additions. 

The  proof  of  the  claim  is  rather  straightforward.  Consider  the  following  two 
constraints  in  S: 

n  n 

^aixt  <  ao  and  ^  a{j 

t=i  i=i 

where  >  0  and  aj  >  0.  The  resulting  constraint  using  the  Fourier-Motzkin 
method  is: 

n 

-  aia'^)  xi  ^  (otoa-  - 

^=1 

Note  that  in  the  above  constraint  the  coefficient  for  Xi  is  0  (hence  Xi  is  elimi¬ 
nated).  The  new  constraint  verifies  the  statement  of  the  claim.  It  is  easy  to  see 
that  for  any  type  of  linear  constraints  the  claim  holds. 

We  now  see  that  the  projection  of  S  can  be  done  in  AC®.  Since  5  is  a  fc- 
bounded  set  of  linear  constraints,  the  variable  coefficients  are  not  larger  than  the 
constant  k.  Therefore,  the  resulting  constraints  are  obtained  by  multiplication 
with  a  constant  (integer)  not  larger  than  A;,  and  by  addition.  These  two  opera¬ 
tions  can  be  done  in  AC®  (it  follows  from  Lemma  5.3).  Moreover,  the  number 
of  resulting  new  constraints  is  at  most  quadratic  in  the  number  of  initial  con¬ 
straints,  using  the  Fourier-Motzkin  method.  ■ 

The  only  other  operation  that  requires  some  care  is  the  set  difference.  The 
next  lemma  is  devoted  to  the  complement  operation,  that  can  be  used  to  define 
set  difference. 


r 
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Lemma  5.5  Let  A:  be  a  (fixed)  positive  integer  and  ICk  be  the  class  of  A;-bounded 
linear  constraint  relations.  There  is  a  polynomial  function  V,  such  that  for  each 
relation  R  in  ICk  of  size  n,  the  following  conditions  hold  for  the  complement, 
of  R:  (i)  \R^\  ^  ‘P(n),  and  (ii)  R^  is  computable  in  AC°  in  the  size  of  R, 

Proof:  Assume  that  i?  is  an  r-ary  relation  of  size  n.  Since  i?  is  a  A;-bounded  linear 
constraint  relation,  it  follows  that  the  number  of  different  slopes  of  hyperplanes 
in  R  is  the  number  of  nonnegative  integer  solutions  to  the  equations  +  2:2  + 

/  _  1  I  • 

...  Zr  =  j  where  1  ^  ^  fc  4-  2.  In  particular,  ^  ^ 

7^1  V  ^ 

r  <  k’  ^  0{k^).)  Therefore  each  cell  (in  the  sense  of  [Col75])  can  be  defined 
by  a  tuple  with  no  more  than  2k'  constraints.  Assume  that  R  is  defined  with 
i  different  constraints.  It  can  be  seen  that  the  constraints  needed  to  define 
the  cells  in  the  complement  are  the  existing  constraints,  and  their  variants 
obtained  by  replacing  the  predicate  in  each  constraint  with  one  of  “=”,  “<”,  or 
“>”.  This  generates  at  most  constraints.  Since  no  other  constraint  is  required, 
every  cell  in  the  plane  is  therefore  definable  with  at  most  2k'  constraints.  There 
are  M  possibilities  for  each  constraint,  thus  it  leads  to  at  most  (3^)^*='  possible 
cells.  The  number  of  cells  is  therefore  bounded  by  a  polynomial  function  in  n 
(see  also  [Col75]),  and  the  complement  can  easily  be  computed  in  AC®  (using 
only  operations  in  ALG^  as  shown  in  Example  4.2).  ■ 

We  are  now  ready  to  prove  Theorem  5.2. 

Proof  of  Theorem  5.2:  The  proof  is  by  induction  on  the  structure  of  the  for¬ 
mula  expressing  the  query.  We  can  always  assume  that  the  boolean  query  is  of 
the  form  7r0e,  where  e  is  some  algebraic  expression  (i.e.  a  test  of  emptiness). 

Basis:  Assume  e  =  R.  To  verify  that  'K0R  is  false,  it  suffices  to  check  that  each 
tuple  in  R  defines  an  empty  set.  This  can  be  done  by  applying  the  Fourier- 
Motzkin  method.  It  follows  from  Lemma  5.4,  that  this  can  be  done  in  AC®. 

In  the  sequel,  we  prove  by  induction  that  we  can  compute  in  AC®,  an  encoding 
of  e{R)  starting  from  an  encoding  of  R  for  each  subexpression  e. 

Induction:  The  induction  step  depends  on  the  last  algebraic  operations  per¬ 
formed.  We  first  consider  the  gates  of  the  circuit,  and  then  illustrate  how  it  is 
wired.  We  next  establish  upper  bounds  on  (i)  the  number  of  constraints  in  each 
tuple,  and  (ii)  the  number  of  tuples  in  the  new  relations,  resulting  from  the 
application  of  algebraic  operations.  Let  be  a  ^-bounded  relation  of  rii  tuples, 
each  tuple  consisting  of  ki  constraints  {i  =  1,2).  Note  that  both  the  number  m 
of  tuples  and  the  number  ki  of  constraints  may  not  be  exact.  However  they  are 
upper  bounds.  Physically,  the  circuits  contain  the  space  to  encode  rij  tuples  of 
ki  constraints. 

1.  If  e  =  (ei  X  62),  then  e  is  a  /c-bounded  relation  containing  ni  x  n2  tuples, 
each  of  which  is  represented  with  ki  -f  k2  constraints. 
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2.  If  e  =  (cTi?  ei),  then  e  is  a  fc-bounded  relation  containing  ni  tuples,  each  of 
which  is  represented  with  fci  +  1  constraints. 

3.  If  e  =  TT  ei ,  where  tt  just  eliminates  a  single  variable,  then  e  is  a  fc^-bounded 
relation  containing  ni  tuples,  each  of  which  is  represented  with  at  most 
constraints. 

4.  If  e  =  (ei  U  62),  then  e  is  a  A:-bounded  relation  containing  ni  +  n2  tuples, 
each  of  which  is  represented  with  max{ki,k2)  constraints. 

5.  If  e  =  (ei  n  62),  then  e  is  a  A;-bounded  relation  containing  ni  x  712  tuples, 
each  of  which  is  represented  with  ki  +  k2  constraints. 

6.  If  e  =  (ei  -  62),  then  e  is  a  fc-bounded  relation  containing  P(ni,n2,fci, ^2) 
tuples,  each  of  which  is  represented  with  at  most  'P’{ki,k2)  constraints. 
In  the  binary  case  (ei  and  62  binary),  'P{ni,n2,ki,k2)  =  ni  x  {3n2k2)^^ 

and  'P'{kiik2)  =  maa:(fci,  2/?'^),  where  k'  =  ^  —  1.  For  larger 

arities,  both  the  number  of  tuples  and  the  number  of  constraints  per  tuple 
are  bounded  by  similar  polynomials. 

The  above  follows  from  the  definition  of  the  algebraic  operations  in  Section  4, 
from  Lemma  5.4  for  the  case  of  the  projection,  and  from  Lemma  5.5  for  the 
case  of  the  set  difference.  In  this  last  case,  (ei  -  62)  =  (ei  n  e^),  where  63  is 
a  A;-bounded  relation  containing,  in  the  binary  case,  a  maximum  of  (3n2A;2)^^ 
tuples,  each  having  at  most  k'^  constraints. 

It  follows  that  the  number  of  tuples  and  the  number  of  constraints  in  each 
tuple  are  bounded  by  some  polynomial  function.  Note  that  the  integers  occurring 
in  the  constraints  during  the  computation,  come  either  from  the  input,  from  the 
query,  or  result  from  a  projection.  One  projection  generates  quadratic  numbers, 
and  so  their  binary  representation  has  twice  the  initial  space.  Therefore,  the  size 
of  integers  is  linear  in  n.  For  each  algebraic  sub-expression  of  the  query  Q,  and 
each  tuple  t  of  the  adequate  form  obtained  as  described  above,  we  associate  a 
series  of  gates  encoding  the  pair  Q,tm  the  circuit.  Other  gates  are  also  required 
in  computing  the  additions  of  constants  for  the  new  constraints  resulting  from  the 
projection  operator.  As  shown  in  Lemma  5.4,  there  are  only  a  polynomial  amount 
of  these.  The  selection  also  requires  built-in  gates  to  encode  the  constraint  in 
the  selection  itself.  This  is  easily  done  with  a  number  of  gates  bounded  by  a 
constant  in  the  size  of  the  query.  Essentially,  no  more  gates  are  needed  to  encode 
the  whole  circuit.  It  follows  that  the  number  of  gates  needed  to  encode  the  whole 
computation  is  also  polynomially  bounded. 

We  now  see  how  the  wires  between  the  gates  previously  presented,  can  be 
uniformly  defined. 

The  algebraic  operations  have  various  effects  on  their  inputs.  They  can  mod¬ 
ify  (or  rearrange)  the  initial  tuples  and/or  the  constraints,  (i)  The  union  opera¬ 
tion  changes  only  at  the  relation  level  and  the  initial  tuples  remain  unchanged, 
(ii)  Cartesian  product,  selection,  and  intersection  create  new  tuples  from  old 
tuples,  by  using  the  initial  constraints  which  are  not  changed,  (iii)  Set  differ¬ 
ence  creates  new  constraints  obtained  from  old  constraints,  by  just  changing  the 
predicates  in  the  constraints,  (iv)  Projection  creates  new  constraints,  with  new 
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parameters  as  shown  in  Lemma  5.4. 

In  the  case  of  U,  D,  x ,  and  cp,  it  is  clear  that  the  new  tuples  can  be  computed 
easily  in  AC°.  More  precisely,  these  operations  result  only  in  a  reorganization 
of  existing  constraints  inside  the  tuples,  and  of  tuples  inside  the  new  relations. 
The  wires  are  essentially  used  to  copy  values  (with  no  computation).  They  do 
not  have  to  be  materialized. 

Two  operators,  projection  and  set  difference,  deserve  a  more  thorough  ex¬ 
amination.  Indeed,  they  result  in  the  definition  of  new  constraints,  with  new 
parameters  obtained  from  old  parameters  by  addition,  or  iterated  addition.  It 
follows  from  Lemmas  5.4  and  5.5,  that  the  two  operations  can  be  computed  in 
uniform  AC°.  ■ 

Kanellakis  and  Goldin  [KG95]  suggested  to  study  the  data  complexity  of 
first-order  queries  over  linear  constraint  databases  in  the  case  where  integers  are 
encoded  in  unary.  In  the  remainder  of  the  section,  we  briefly  discuss  the  data 
complexity  of  FO^  for  arbitrary  linear  constraint  databases,  i.e.  without  the 
restriction  of  being  A;-bounded.  We  show  that  under  the  unary  representation  of 
integers,  the  data  complexity  remains  in  AC°. 

Let  m  be  the  circuit  input  size.  The  unary  representation  of  an  integer  n 
is  a  string  amam~i  •  •  •  02^1  where  aj  =  1  for  each  1  ^  i  ^  n  and  Oj  =  0  for 
n  <  i  ^  m.  We  now  show  that  the  addition  and  the  multiplication  of  two 
integers  encoded  in  unary  representation  can  be  done  by  boolean  circuits  of 
constant  depth  (i.e.  in  AC®). 

Theorem  5.6  Let  m  G  N.  The  addition  (and  multiplication)  of  two  (positive) 
integers  la,  h  such  that  la  +  h  (resp.  la  x  If,  ^  m)  in  unary  representation, 
• "  02O1  and  5m&m-i  *  *  *  can  be  done  by  constant-depth  circuits 
with  m  output  gates  and  polynomially  (in  m)  many  gates. 

Proof:  We  first  consider  addition.  Let  /<,  =  +  h  and  CmCm-i  *  •  •  C2C1  be  the 

unary  representation  of  /c*  It  is  observed  that  Ic  can  be  computed  by  counting 
all  gates  from  ■  02^1  and  bmhm~i'”b2hi  that  are  true.  Indeed,  for 

each  1  <  i  <  m,  Ci  can  be  defined  by  the  following  boolean  function  (note 
that  a j  =  1  =  1  for  each  i  ^j): 


/  i-i 

/j  =  flt  V  I  (^i—j 


Vbi 


It  is  easy  to  see  that  fi  can  be  realized  by  a  circuit  of  depth  no  more  than  2  and 
of  no  more  than  (2m  —  1)  gates. 

Now  let  Ic  —  la  X  If,  be  the  product  and  we  assume  again  c,nCm-i  •  •  •  C2C1 
is  the  unary  representation  of  Ic.  Then,  the  multiplication  can  be  viewed  as  the 
following  sum  of  integers  in  unary  representation: 


E 


®m®m— 1  *  *  ‘  Q'2ai 
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Thus,  it  is  easy  to  see  that  for  each  1  ^  i  ^  m,  c*  is  defined  hy  the  following 
boolean  function  gi'. 

9i  =  'yi  (Oj  A  6*) 

where  the  condition  r?(j,  k)  states  that  1  ^  ^  m,  x  fc  ^  and  both  (j  —  1)  x  A; 

and  j  X  (fc  —  1)  are  <  i.  Hence,  the  circuit  realizing  gi  has  depth  2  and  number 
of  gates  linear  in  m.  Note  that  for  each  m  G  N,  the  circuits  /i,  -  /mjffi? 
can  be  uniformly  constructed.  Therefore,  addition  and  multiplication  of  integers 
in  unary  representation  are  in  AC°.  ■ 

Since  both  addition  and  multiplication  of  integers  in  unary  representation 
can  be  computed  by  circuits  of  constant  depth,  it  can  be  verified  that  the  data 
complexity  of  FO^  over  linear  constraint  databases  remains  in  AC®  under  the 
unary  encoding  assumption.  The  size  of  the  numbers  that  are  derived  by  a  query 
from  the  numbers  in  the  input  is  defined  by  a  polynomial  which  depends  only 
upon  the  query  itself,  and  enough  space  is  devoted  to  them  in  the  circuit.  The 
proof  follows  the  same  lines  as  the  proof  of  Theorem  5.2.  The  assumption  of 
fc-boundedness  was  needed  in  Theorem  5.2  to  prove  that  projection  involved 
only  multiplication  by  a  constant.  This  assumption  is  not  needed  here  since 
multiplication  of  unary  numbers  can  be  done  in  AC®.  For  the  set  difference 
operation,  using  multiplications  it  is  possible  to  “triangulate”  the  plane  (when 
the  arity  is  2)  or  hyperplane  (when  the  arity  is  higher)  using  the  constraints  in  the 
input.  Thus  to  compute  the  complement,  one  needs  to  consider  only  tuples  with 
up  to  a  fixed  number  (depending  only  on  the  arity)  of  constraints  (3  constraints 
when  the  arity  is  2).  We  can  use  an  approach  for  computing  the  complement 
and  set  difference  similar  to  the  one  described  in  the  proof  of  Lemma  5.5  and 
the  fc-boundedness  assumption  is  not  necessary. 

In  the  next  section,  we  examine  consequences  of  the  complexity  upper  bound 
on  the  expressive  power  of  linear  constraints. 

6  Expressive  Power 

In  this  section,  we  study  the  expressive  power  of  first  order  query  languages 
for  linear  constraint  databases.  In  particular  we  consider  queries  from  relational 
database  theory  (parity),  graph  theory  (graph  connectivity),  and  geometry  (re¬ 
gion  connectivity)  and  show  that  these  queries  are  not  first  order  expressible. 
The  proof  of  these  results  uses  the  AC®  upper  bound  on  data  complexity  (Theo¬ 
rem  5.2)  and  first  order  reductions  from  boolean  functions,  such  as  PARITY  which 
is  known  to  be  outside  AC®  [FSS84]. 

Let  (T  =  {i?}  be  a  signature  where  i?  is  a  unary  relation  symbol.  For  a 
database  instance  I  of  cr,  the  parity  query  answers  “yes”  if  I{R)  is  finite  and 
has  an  even  cardinality.  The  graph  connectivity  query  is  defined  over  a  signature 
consisting  of  a  single  binary  relation  G.  The  query  answers  “yes”  on  an  instance 
I  if  I{G)  is  a  connected  finite  graph.  For  the  third  example,  we  consider  the 


444 


k- dimensional  region  connectivity  query  over  possibly  infinite  input  instances, 
where  k'^1.  The  query  is  also  a  boolean  query  and  answers  “yes”  on  an  instance 
I  if  every  pair  of  points  in  I{R)  can  be  linked  by  a  continuous  curve  lying  entirely 
in  I{R).  Note  that  for  A;  =  1,  the  query  can  be  easily  expressed. 

Theorem  6.1  The  following  queries  are  not  definable  in  {=,^,H-}UQ: 

1.  Parity  of  cardinality, 

2.  Graph  connectivity, 

3.  A;-dimensional  region  connectivity  for  each  k'^  2. 

Proof:  By  Theorem  5.2,  it  is  sufficient  to  show  that  these  queries  are  not  in  AC°. 
We  first  consider  the  parity  query  and  describe  a  straightforward  reduction  from 
the  boolean  function  parity.  The  parity  function  takes  n  boolean  inputs  and 
returns  “true”  if  the  number  of  inputs  equal  to  1  is  even.  Now  let  xi, . . .  be 
the  n  inputs  for  parity.  We  construct  a  database  I  over  the  signature  with  one 
unary  relation  symbol  R  as  follows:  I(R)  =  {i  |  xi  =  1}.  Clearly  the  database  is 
definable  using  only  equality  constraints,  without  the  addition  symbol.  In  other 
words,  I  is  in  /Co  (0-bounded).  Obviously,  the  construction  can  be  done  in  first 
order  and  PARITY(xi,  ..,,x„)  =  1  iff  the  parity  query  on  I  answers  “yes”.  For 
graph  connectivity,  we  use  the  classical  reduction  from  the  parity  query.  Let  I 
be  an  input  instance  of  the  parity  query  and  G  be  a  binary  relation  symbol. 
Suppose  I{R)  =  {ai, ...,  fln})  and,  without  loss  of  generality,  ai  <  02  <  •  •  •  <  Un- 
We  define  an  instance  J  over  G  as  follows.  Let  J{G)  be  the  symmetric  closure 
of  the  set  {(01,0^)}  U  {(ai,at+2)  |  1  <  ^  <  n  -  2}.  It  is  easy  to  verify  that 
parity  on  I  answers  “yes”  iff  J{G)  is  connected.  Finally,  for  region  connectivity 
in  dimension  A:  ^  2,  it  is  shown  in  [GS95]  that  it  is  not  in  AC®,  by  a  reduction 
from  the  boolean  function  MAJORITY.  ■ 

The  previous  result  can  be  generalized  to  various  contexts. 

Corollary  6.2  The  queries  of  Theorem  6.1  are  not  definable  with  linear  con¬ 
straints  over  the  following  domains:  the  natural  numbers,  N,  the  integers,  Z,  the 
rationals,  Q,  and  the  reals,  R. 
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Abstract.  We  define  and  justify  a  natural  sequential 
model  of  computation  with  a  constant  amount  of 
read/write  work  space,  despite  unlimited  (polynomial) 
access  to  read-only  input  and  write-only  output.  The 
model  is  both  deterministic,  uniform,  and  sequential. 
The  constant  work  space  is  modeled  by  a  finite  number 
of  destructive  read  boolean  variables,  assignable  by 
formulas  over  the  canonical  boolean  operations.  We 
then  show  that  computation  on  this  model  is  equivalent 
to  expressibility  in  first-order  logic,  giving  a  duality 
between  (read-once)  constant-space  serial  algorithms 
and  constant-time  parallel  algorithms. 


§  0  Introduction 
Summary 

Problems  computable  in  constant  time  on  a  uniform  parallel  model  of  computation  (a 
type  of  PRAM)  have  been  elegantly  characterized  as  those  expressible  in  first-order  logic 
(FO)  on  binary  strings  [I].  It  is  also  known  that  FO  is  identified  with  LH,  the  logtime  ^ter- 
nation  hierarchy  based  on  random-access  Turing  machines  IBIS].  We  provide  an  additional 
correspondence  between  FO  and  those  problems  computable  in  constant  space  on  a  deter¬ 
ministic  sequential  model  of  computation. 

The  key  to  this  idea  is  a  very  careful  measuring  of  work  space  in  a  machine.  Ordinarily, 
read-only  input  and  write-only  output  are  not  considered  part  of  the  read/write  work  space. 
This  is  an  essential  concept  for  defining  the  complexity  class  L  (logarithmic-space).  We  go 
somewhat  farther  in  our  model,  and  do  not  include  in  the  work  space  any  storage  mechan¬ 
ism  required  to  access  the  input  or  output,  be  it  memory  addressing  or  tape  scanning.  By 
making  the  access  scheme  oblivious,  we  are  careful  not  to  let  the  machine  cheat  by  using  the 
memory  addresses  or  head  positions  as  read/write  storage. 

Furthermore,  we  take  the  additional  step  of  separating  the  flow  of  control  of  the 
machine  from  the  computation  it  is  performing.  Specifically,  we  imagine  a  machine  con¬ 
trolled  by  a  simple  programming  language  with:  a  finite  set  of  read/write  boolean  variables; 
the  operations  AND,  OR,  NOT;  composition  of  program  statements;  and  a  strict  form  of 
definite  loops.  No  conditionals  are  allowed  in  the  programming  language  (if...then,  or 
while ...  repeat)  to  insure  the  oblivious  nature  of  the  computation.  In  addition,  we  impose  a 
read-once  (destructive  read)  condition  that  prevents  a  read/write  boolean  vari^le  from  being 
read  more  than  once  without  an  intervening  write.  No  such  restriction  applies  to  the  input 
or  output  however. _ 

t  Partially  supported  by  NSF  grant  CCR-9403447,  and  the  John  C.  Whitehead  faculty  research  fund  at 
Haverford  College.  Electronic  mail  address:  <slindell@haverford.  edu>. 
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The  following  is  representative  of  our  main  theorem: 

Theorem:  A  query  on  binary  strings  is  first-order  definable  if  and  only  if  it 
is  computable  by  an  constant- space  read-once  serial  algorithm. 


Motivation 

Classically,  when  defining  sub-linear  space  on  a  Turing  machine,  we  resort  to  an  off¬ 
line  model  which  separates  read-only  input  and  write-only  output  from  the  read/write  work 
tape.  In  this  way  we  can  get  robust  definitions  of  SPACE(log  n)  and  above.  And  although 
logspace  transducers  have  proved  to  be  a  useful  reducibility  between  problems,  a  finer 
notion  of  reduction  based  on  first-order  translations  has  led  to  some  illuminating  results:  a 
very  restricted  version  of  the  Berman-Hartmanis  conjecture  [ABI];  and  a  very  deep  result 
concerning  the  recursive  enumerability  of  the  polytime  queries  [D'].  It  has  also  been  shown 
that  first-order  logic  provides  a  robust  notion  of  uniformity  for  the  study  of  the  fine  struc¬ 
ture  of  low-level  circuit  complexity  classes  [BIS],  and  the  corresponding  first-order  reduc¬ 
tion  has  been  shown  equivalent  in  [AG]  to  a  much  earlier  notion  of  logspace  rudimentary 
reductions.  Furthermore,  first-order  translations  are  based  on  the  classical  notion  of  inter¬ 
pretation  as  spelled  out  in  [E],  and  serve  as  excellent  reductions  which  preserve  the  comple¬ 
teness  of  well-known  NP-problems  [D]  as  well  as  newer  ones  (the  boolean  formula  value 
problem  being  complete  for  ALOGTIME)  [B]. 

Finite-state  transducers  fail  in  this  capacity  because  they  cannot  do  the  arithmetic  needed 
to  convert  binary  input  from  one  simple  form  into  another  (like  reversing  a  string),  nor  can 
they  provide  polynomial  magnification  (required  for  the  existence  of  complete  problems). 
This  happens  in  any  standard  off-line  model  with  space  below  0(log  n). 

Our  sequential  model  of  computation  is  able  to  lift  these  limitations  while  still  operating 
under  a  form  of  constant-space  constraint.  This  is  because  our  model  permits  multiple  uni¬ 
directional  heads  which  can  re-scan  the  tape,  but  forbids  head  movement  which  is  non- 
oblivious.  We  will  measure  the  actual  2unount  of  read/write  work  space  in  a  very  careful 
fashion.  In  particular,  our  model  does  not  even  count  any  space  used  to  access  the  input, 
whether  it  be  a  head  scanning  a  tape,  or  an  address  into  random-access  memory,  t  The  rea¬ 
soning  for  this  is  in^itive:  if  the  input  (output)  tape  is  read-only  (wiite-only),  and  the  access 
to  the  tape  is  oblivious,  then  the  machine  cannot  use  the  tape  head  as  a  read/write  storage 
mechanism,  so  that  space  does  not  count.  Clearly  this  intuition,  if  correct,  extends  to  any 
fixed  number  of  heads.  If  access  to  the  tape  is  not  oblivious,  and  we  allow  2- way  arbitrary 
movement  of  the  head  in  a  finite  automata,  then  all  of  logspace  is  achieved  ([G],  p.  53).  In 
fact,  our  model  will  be  explained  in  terms  of  a  programming  language,  much  the  same  as 
the  presentation  of  primitive  recursion  in  [ibid,  p.20],  and  can  be  compared  with  the  uni¬ 
form  constant- width  circuits  of  [BI,  section  6],  which  gives  a  corresponding  characteriza¬ 
tion  of  uniform-NCk  Also,  [C]  has  given  machine-independent  algebraic  characterizations 
of  ACO  and  various  other  small  complexity  classes  using  sequential  operations. 

Overview 

Section  1  provides  a  brief  review  of  first-order  definable  queries,  including  examples. 
Of  particular  importance  is  the  discussion  of  binary  string  structures  and  the  special  numeri¬ 
cal  predicates  for  arithmetic.  Section  2  defines  and  justifies  the  constant-space  model  of 
sequential  computation  that  this  paper  introduces,  comparing  it  with  the  classic^  definition 
of  finite-state  automata.  Section  3  illustrates  this  model  with  two  contrasting  bit-serial 
examples:  parity;  and  addition.  Section  4  contains  the  main  result,  mentioned  above,  and  its 
proof.  The  remainder  of  the  paper,  section  5,  concludes  by  describing  possible  directions 
for  future  research. 

t  However,  it  will  be  prefer^le  to  use  a  cursor  to  mark  head  position,  since  this  method  of  memory  access 
appears  aesthetically  more  sequential,  as  opposed  to  RAM  which  involves  an  implicit  use  of  parallelism 
(address  decoding). 
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§  1  Background 
First-order  Queries 

One  way  of  mathematically  studying  the  computational  complexity  of  combinatorial 
problems  is  to  directly  examine  how  difficult  it  is  to  define  them.  Instead  of  measuring 
asymptotic  resources  (such  as  time  or  space)  required  to  compute  a  problem,  one  c^  clas¬ 
sify  problems  as  to  the  power  of  the  logic  required  to  express  their  solution.  Specifically, 
an  input  instance  is  2l  finite  relational  structure, 

(A,  Ru  Rh  Ch  Cl) 

consisting  of  a  finite  set  A,  called  the  domain,  together  with  relations  Ri  (each  of  a  speci¬ 
fied  arity  on  A),  and  constants  cj  (individual  elements  of  A).  The  output  is  determined  by  a 

query,  a  global  relation  across  structures  of  the  same  type  (signature),  mapping  each  one  to 
a  relation  (of  fixed-arity)  on  the  structure.  One  of  the  simplest  “languages”  for  expressing 
queries  is  that  of  first-order  logic.  Formulas  in  first-order  logic  permit:  individual  variables 
interpreted  as  ranging  over  the  domain;  constant  symbols  for  each  constant  cj,  predicate 
symbols  for  each  relation  Ri  and  equality  (=);  the  Boolean  connectives  a,  v,  and  — i;  and 
quantification  of  the  variables.  We  use  FO  to  denote  the  class  of  all  queries  determined  by 
such  first-order  formulas.  For  further  background,  see  [E]. 

Simplicity  Example 

One  of  the  easiest  and  most  familiar  examples  of  finite  structures  is  the  class  of  directed 
graphs  —  all  structures  having  a  binary  edge  relation  E  over  a  finite  domain  of  vertices  V: 

G  =  {V,E)  EcV2 

The  problem  of  determining  if  a  graph  is  simple  (no  self-loops,  all  edges  midirected)  is 
an  example  of  a  graph  property,  or  boolean  query  of  arity  0  which  is  expressible  as  a  first- 
order  sentence. 


G  =  {V,E)  is  simple  iff  G  |=  0  where 

0  =  -.{3x)[E(x,  X)]  A  (Vy)(Vz)[E(y,  z)  E{z.  y)] 

The  first  part  of  0  says  that  no  vertex  has  an  edge  to  itself,  and  the  second  part  says  that  if 
there  is  an  edge  from  one  vertex  to  another,  then  there  must  be  an  edge  going  the  opposite 
direction. 

Ordering  Example 

Another  example  is  the  problem  of  determining  if  a  binary  relation  constitutes  a  total  lin¬ 
ear  order  of  the  vertices.  Using  more  suggestive  notation 

B  -  (A,  <)  is  ordered  iff  5  |=  Xj/ 

where  \|/  is  the  conjunction  of  the  following  universally  quantified  axioms 
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(iireflexivity) . 

(totality) 

(transitivity) 


-i(x  <  x) 

(Jf9ty)-^  [(jr<y)v(y<x)] 

[{x<y)  A  (y<  z)]  ~^(x<z) 

Binary  String  Structures 

Ordering  the  Positions 

Although  graphs  universal  (as  general  as  any  other  type  of  structures)  for  express- 
ibility  purj^ses  [LI],  it  seems  necessaty  to  work  on  ordered  structures  to  express  computa¬ 
tions,  particularly  the  contents  of  the  input  as  it  is  presented  to  a  machine.  Modem  digital 
computers  use  binary  strings  for  I/O,  and  we  can  represent  these  in  the  form  B  = 
({0, 1, ...,  n  -  1 },  <,  C/)»  where  the  domain  is  the  set  of  positions  in  the  string,  <  orders 
these  positions  from  left  to  right  in  the  usual  fashion  0<1<...<  n-l,and  C/ indicates 
where  the  I’s  and  O's  are  by  tme  and  false,  respectively.  For  instance,  the  binary  string 
1010  is  represented  by  the  stmcture  ({0, 1,  2,  3},  <,  {0,  2}),  with  0  <  1  <  2  <  3. 

•  <  O  <  •  <  O 

0  12  3 

In  the  figure,  closed  circles  indicate  where  U  is  tme,  and  open  circles  where  U  is  false. 

In  general,  given  a  binary  string  we  {0, 1  }*,  we  create  a  canonical  stmcture  for  it: 

<|w|,  <,  {  / :  wi  =  1}), 

where  |w|  is  the  length  of  w,  and  wi  is  the  symbol  of  w. 

Adding  Arithmetic 

To  accurately  capture  fine  notions  in  resource-bounded  computation  (such  as  parallel 
time)  appears  to  require,  in  addition  to  the  ordering,  a  method  whereby  the  binary  input  to  a 
machine  can  be  accessed  effectively  [I].  For  this  purpose  (what  might  be  called  address 
arithmetic)  it  suffices  to  have  a  special  predicate  which,  for  each  domain  element  i,  gives  the 
location  of  1* s  and  O’s  in  its  binary  representation: 

<=> 

the/h  position  in  the  binary  representation  of  Hs  a  1  .t 

For  instance,  bit(5,  1)  is  false,  since  5  =  (101)2,  ^uid  there’s  a  0  in  the  first  position  (the 
rightmost  bit  is  treated  as  the  zeroeth  position).  This  leads  us  to  the  following  definition. 

Definition:  For  each  we  {0, 1  }*,  define  the  binary  string  structure  for  w  to  be: 


Aw  =  <k|,  <,  bit,  {/ :  Wi  =  1}). 


We  distinguish  the  correspondingly  augmented  class  of  first-order  queries  by  the  notation 
FO(<,  bit),  indicating  that  <  and  bit  are  assumed  to  be  “givens”  in  the  same  way  =  is  taken 
for  granted.  See  [L2]  for  a  brief  discussion  that  this  in  fact  is  equivalent  to  first-order  logic 
with  arithmetic  on  the  domain,  FO(+,  *,  ^). 


t  For  a  discussion  of  the  logical  importance  of  the  bit  predicate,  see  [T.9.], 
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§  2  The  Machine  Model 

Comparison  with  Regular  Languages 

Before  discussing  the  machine  model,  it  will  be  instructive  to  compare  the  computa¬ 
tional  complexity  of  FO(<,  bit)  with  the  more  familiar  regular  languages,  denoted  REG. 
First  note  that  both  FO(<,  bit)  and  REG  are  strictly  contained  in  ALOGTIME  (=  uniform- 
NCl).  A  classical  result  is  that  the  regular  languages,  viewed  as  collections  of  binary  string 
structures  without  bit,  are  precisely  those  definable  in  monadic  second-order  logic  (see  [S]). 
Equally  important  for  our  purposes  is  the  fact  that  first-order  logic  on  binary  string  struc¬ 
tures  w/o  bit  corresponds  exactly  to  the  star-free  fragment  of  REG.  In  fact,  FO(<,  bit)  Pi 
REG  is  equal  to  the  class  of  first-order  definable  queries  on  binary  strings  with  order, 
together  with  the  unary  numerical  predicates  M]c  =  {tri’k  :  m  =  0,  1,...}  for  each  ^  >  0 
[BCST].  This  is  denoted  FO(<,  mod)  in  the  following  strict  containment  diagram. 


ALOGTIME 


Two  simple  examples  serve  to  illustrate  the  contrast  between  FO(<,  bit)  and  REG. 

PARITY  =  {w  6  {0, 1  }*  :  w  has  an  even  number  of  ones} 
MIDPOINT  =  {0«1«  :  n  =  0,  1,  2...} 

A  trivial  finite  automaton  can  recognize  PARITY.  Yet  [FSS]  show  that  PARITY  cannot  be 
definable  in  FO( — )  even  with  arbitrary  numerical  predicates.  Conversely,  a  trivial 
FO(<,  bit)  formula  can  express  MIDPOINT  (by  using  addition).  However,  MIDPOINT 
is  the  classic  example  of  a  non-regular  language.  For  comparison  purposes,  note  that 

EQUAL  =  {w  G  {0, 1  }*  :  w  has  an  equal  number  of  zeros  and  ones} 

is  neither  in  REG  nor  FO(<,  bit),  but  is  in  ALOGTIME. 

Comparison  with  Finite  Automata 

To  help  explain  the  discrepancies  between  regular  languages  and  first-order  logic,  we 
turn  to  the  table  below  which  shows  the  two  distinct  differences  between  finite  automata 
and  the  sequential  deterministic  model  we  propose. 


finite  automata _ proposed  model 


input/output  access 
flow  of  control 


single  oblivious  scan  multiple  oblivious  passes 
state  machine  restricted  state  machine 
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Multiple  Heads 

Whereas  a  finite  automata  scans  its  input  precisely  once  from  left  to  right,  our  model 
allows  for  multiple  heads,  each  of  which  is  permitted  to  re-scan  the  input  tape.  To  prevent 
positional  information  from  being  used  as  read/write  storage,  we  restrict  their  movements  to 
be  oblivious  (unidirectional  with  reset  to  the  left  edge)  so  that  their  locations  depend  only  on 
the  length  of  the  input.  Moreover,  these  heads  serve  not  only  to  access  the  input,  but  also 
as  read-only  clocks  which  can  measure  execution  time,  and  we  include  a  mechanism  wher¬ 
eby  their  relative  and  absolute  positions  can  be  queried. 

Destructive  Read 

When  a  finite  state  machine  is  implemented  in  hardware,  flip-flops  are  used  to  store  the 
current  state,  while  boolean  gates  combine  this  information  witfi  the  current  input  bit  in  an 
interconnection  network  to  set  the  next  state  which  is  stored  back  in  the  same  flip-flops.  A 
fixed  number  of  gates  and  binary  storage  elements  assure  a  constant-space  resource  bound. 
In  our  model,  gate  types  are  restricted  to  AND,  OR,  NOT,  and  we  insist  that  flip-flops  are 
destructively  read  (making  computations  more  akin  to  iterated  boolean  formulas). 

Definition  of  Model 

While  it  is  certainly  possible  to  continue  in  this  fashion  and  define  our  model  in  terms  of 
time  clocks  and  circuit  diagrams,  it  is  more  convenient  to  describe  our  model  in  terms  of  a 
programming  language,  to  make  serial  algorithms  textually  representable.  By  restricting 
storage  to  (read-once)  boolean  variables,  the  software  will  naturily  constrain  the  model  to  a 
constant  amount  of  (destructive-read)  space.  By  limiting  the  constructs  to  composition  and 
a  strict  form  of  definite  loops  (indexed  by  tape  heads),  oblivious  flow  of  control  and  input 
access  will  be  guaranteed. 

We  take  a  multi-head  machine  M  consisting  of  an  input  tape  together  with  a  fixed 
number  of  heads  to  scan  the  tape.  It  is  equipped  with  head-crossing  detectors  which  keep 
track  of  the  relative  left-right  positions  of  any  pair  of  heads,  and  (resettable)  binary  counters 
which  keep  track  of  the  absolute  position  of  each  head.  A  mechanism  is  built-in  whereby 
any  particular  bit  of  a  counter  can  be  queried. 

The  inachine  itself  is  controlled  by  a  sequential  program  P,  whose  syntax  assures  that 
the  machine  obeys  the  polynomial-time  and  constant-space  resource  bounds,  and  whose 
semantics  assure  the  read-once  restriction  and  oblivious  head  movement.  We  define  these 
very  simple  programs  by  induction. 

Booleans 

The  basic  data  type  is  a  boolean  and  only  boolean  variables  are  available  for  read-write 
storage.  At  any  point  in  time,  a  boolean  variable  is  either  in  the  read  or  unread  state. 
Apart  from  variables,  there  are  other  boolean  values  directly  accessible  in  the  machine 
model  which  do  not  have  read  restrictions.  These  are  called  direct  values,  which  come 
from  the  input  (using  any  tape  head  h  as  index),  comparing  the  positions  of  any  two  tape 
heads,  or  querying  any  counter  bit  of  a  tape  head.  They  are  summarized  in  the  list  below. 

h  a  read-once/write  boolean  variable 

I  [h]  false  if  h  is  currently  positioned  over  a  zero  on  the  tape,  true  otherwise 

i  <  J  a  comparison  test  which  yields  true  iff  head  i  is  to  the  left  of  heady 

bit  {i,  j)  true  iff  the yth  column  in  the  binary  counter  for  head  i  is  seti 

When  combining  these,  only  the  operations  of  AND,  OR,  NOT  are  allowed  in  forming  boo¬ 
lean  expressions. 

1  Purists  may  object  to  this  because  the  standard  implementation  of  a  binary  counter  requires  more  than 
0(1)  operations  per  "cycle”,  though  the  number  of  operations  amortized  over  an  entire  loop  is  0(1).  In  any 
event,  this  is  similar  to  the  read-only  clock  in  section  6  of  [BI]. 
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Assignments 

b  is  3.  boolean  variable,  and  e  is  a  boolean  expression  all  of  whose  boolean  variables 
are  distinct  and  unread,  then 

b  :=  e; 

assigns  the  value  of  e  to  b.  Every  boolean  variable  in  e  becomes  read,  and  the  status  of  b 
becomes  unread  (regardless  of  the  state  it  was  in  before).  Taken  together,  these  conditions 
syntactically  prevent  a  boolean  variable  from  being  read  twice  without  an  intervening  write. 
Just  note  that  this  read-once  condition  can  also  be  syntactically  enforced  in  the  further  con¬ 
structions  below,  even  though  we  do  not  indicate  precisely  how  it  is  done  (essentially,  keep 
track  of  which  variables  are  required  to  be  unread  upon  entering  a  block,  and  which  are 
unread  upon  exiting  a  block). 

Composition 

If  P  and  Q  are  programs,  then  so  is  their  sequential  composition. 


P; 

Q; 


provided  the  read-once  condition  is  not  violated. 

Loops 

Tape  heads  can  serve  as  guarded  parametrized  controls  for  a  loop.  If  the  read-once  con¬ 
dition  is  not  violated,  then  the  looping  construct 

LOOP  h 
P; 

binds  tape  head  h  to  scan  the  input  tape  from  the  first  cell  to  the  last,  performing  one  itera¬ 
tion  of  P  for  each  such  position  0,  ...,  n-1,  while  moving  h  on  the  tape  from  the  begin¬ 
ning  (left)  to  the  end  (right).  The  position  of  h  is  not  allowed  to  change  inside  P  (assume 
for  simplicity  that  loop  heads  are  not  re-bound,  i.e.  not  nested  with  the  same  name).  Also, 
assume  that  h  returns  to  the  left  edge  after  completing  the  loop. 

Acceptance 

By  designating  one  of  the  boolean  variables  as  the  answer ^  we  can  define  string  accept¬ 
ance. 

Definition:  Let  P  be  a  read-once  constant-space  sequential  progreun  as  described  above. 
We  say2  (P,  accepts  w  if  running  program  P  on  input  w  with  initial  head  positions 
(range:  {0,  . . .,  |wl  -  1 })  starting  at  h  results  in  a  true  answer.  Omission  of  any  or  all  of 
the  positions  h  implies  those  heads  begin  at  the  left  end  of  the  tape  (position  zero). 

The  corresponding  language  determined  by  P  is  defined  in  the  usual  way. 


2  We  use  underline  to  notate  tuples. 
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§  3  Examples 

We  illustrate  constant-space  programs  by  two  serial  algorithms  to  provide  both  a  coun¬ 
terexample  and  example  to  die  read-once  condition. 

Parity 

This  simple  program  computes  the  parity  of  an  input  string  a(  1 ). .  .a(n). 

p  :=  false  {parity  initially  zero} 

LOOP  h  (from  LSB  to  MSB) 

p  :=  a{h)  XOR  p  {exclusive -or) 

But  the  boolean  variable  p  violates  the  read-once  condition  because  it  must  be  read  twice 
when  forming  the  exclusive-or  from  the  canonical  base  of  boolean  operations  (although 
a(h)  may  be  read  any  number  of  times  since  it  is  a  read-only  input). 

Addition 

Contrast  this  with  the  schoolbook  algorithm  for  serial  binary  addition  of  two  /i-bit 
numbers,  a(n)...a(l)  +  which  yields  an  n-bit  sum  j(7i)...5'(l)  and  a  carry.  It 

requires  a  single  read/write  boolean  variable  c  for  the  carry: 

c-  :=  0  {initialize  carry} 

LOOP  h  {from  LSB  to  MSB} 

s{h)  a(jh)  XOR  b{h)  XOR  c  {see  notes} 

c  :=  a{h)  AND  b{h)  OR  c  AND  {a{h)  OR  b{h)) 


Note  how  a  simple  re-pareiithesization,  in  the  last  line,  of  the  3-input  majority  function 
(which  is  all  the  carry  really  is)  results  in  a  program  obeying  the  read-once  condition.  Also, 
note  well  that  the  reads  of  c  used  in  assigning  s(h)  do  not  count  since  the  destination  is 
wnte-only.  (W e  have  not  included  output  statements  in  this  preliminary  draft  of  the  paper). 

A  consequence  of  our  main  theorem  will  be  that  this  constant-space  serial  algorithm 
implies  the  existence  of  a  constant-time  parallel  algorithm  for  n-bit  binary  addition,  which  I 
find  quite  surprising  by  itself,  since  the  standard  carry  look-ahead  algorithm  was  not  a  com¬ 
pletely  trivial  observation  in  its  time.  Conversely,  the  existence  of  a  constant-time  parallel 
algorithm  for  binary  addition  implies  the  existence  of  a  (read-once)  constant-space  serial 
algorithm.  This  phenomenon  of  time-space  duality  will  be  discussed  further  in  §  5. 

§  4  Main  Result 

Theorem:  A  binary  language  L  c  { 0, 1  }*  is  recognized  by  a  read-once  constant-space 
sequential  program  P  if  and  only  if  it  is  definable  by  a  first-order  sentence  ^  over  the  class 
of  binary  string  structures.  I.e.  we  L  iff  A^  |=  (j). 


Proof:  (<=)  We  show  by  induction  over  the  quantifier  depth  d  of  sl  first-order  {<,  bit,  U]- 
formula  (p(^  in  prenex  form,  that  there  is  a  program  P  such  that 

A>vl=(p[AI  <=>  accepts  w 

where  h  is  a  tuple  of  numbers  between  0  and  n  -  1,  whose  length  equals  the  length  of  x. 
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Basis 

If  (p(i)  is  quantifier-free,  then  it  is  easy  to  see  that  the  boolean  formula  determined  by 
(pffel  can  be  computed  by  a  loop-free  program,  since  for  all  i  and  j  in  h,  the  atomics  i  <  j 
and  bit(i,  j)  are  built-in  direct  values  of  the  same  name  in  the  machine  model,  and  because 
UQi)  can  be  directly  read  off  the  input  by  I  [h] ,  since  the  tape  head  assigned  to  h  starts  on 
that  square  to  begin  with  by  assumption. 

Induction 

Suppose  (p(^  =  (Qy)V(y,  where  Qg  {3,  V}.  By  induction  hypothesis, 
\^[k,  h}  is  computed  by  a  program  (P,  (k,  h))  To  compute  (p[h]y  we  loop  the  first  head 
around  the  program  P,  and  add  an  additional  variable  to  store  and  compute  the  result  of  the 
quantification.  Here  is  the  program  for  existential  quantification: 

Jb  :=  false 
LOOP  k 
P; 

b  :=  Jb  OR  (the  answer  from  P) 
cinswer  b 

Note  that  program  P  must  be  repeatedly  run  for  each  position  L  A  similar  dual  program 
can  be  used  for  universal  quantification. 

(=>)  The  reverse  direction  is  considerably  more  difficult.3  We  will  express  a  read-once  pro¬ 
gram  P  by  a  collection  of  first-order  formulas,  IT  =  [nbi^d  :  b  e  P]  (but  from  now  on 
dropping  the  clarification  be  P  since  it  will  be  obvious  from  context),  such  that  for  each 
boolean  variable  bof  P  (left  unread  upon  exit),  the  final  result  left  in  b  after  running  P  with 
initial  head  positions  h  on  input  w  is  the  same  as  the  truth  value  of  1=  TibUi]^  Boolean 
variables  will  appear  as  nullary  atomic  relations  in  these  formulas,  and  furthermore,  we  will 
guarantee  that  each  one  occurs  at  most  once  in  IT,  calling  this  the  single  occurrence  proper¬ 
ty' 

Taking  advantage  of  the  definability  of  arithmetic  in  FO(<,  bit),  we  will  freely  use  addi¬ 
tion  and  multiplication  for  computations  in  the  formulas  we  construct.  As  before,  initial 
head  positions  will  correspond  to  free  variables  in  formulas.  Once  these  are  fixed,  a  pro¬ 
gram  P  can  be  thought  of  as  a  map  from  booleans  to  booleans.  Our  proof  continues  by 
syntactic  induction  on  P. 

Base  Case:  P  is  a  single  assignment  statement:  b  :=  e;. 

In  this  case,  e  is  a  boolean  expression  of  direct  values  and  boolean  variables.  Hence  P 
can  be  represented  quite  easily  by  the  single  formula  nb  =  e,  where  the  comparison  and 
binary  calculation  of  head  positions  are  by  definition  the  atomic  relations  <  and  bit,  and  any 
references  to  input  values  I  [b]  are  replaced  by  the  atomic  VQi).  It  remains  to  check  that 
%b  satisfies  the  single  occurrence  property,  but  this  is  clearly  the  case  since  each  boolean 

variable  occurs  at  most  once  in  e  in  order  for  P  to  satisfy  the  read-once  condition. 

Inductive  Steps: 

Consider  the  program  P;  0; .  By  induction  hypothesis,  there  is  a  collection  of  first- 
order  formulas  n  =  {it/;}  expressing  P,  and  a  similar  collection  0  =  {6^,}  expressing  Q, 
each  satisfying  the  single  occurrence  property.  If  a  boolean  variable  aofQ  does  not  appear 


3  Even  more  difficult  than  we  will  make  it  appear.  See  the  coda  at  the  end  of  this  proof  for  an  explanation. 
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in  P,  then  adjoin  to  n  the  identity  formula  Uq  ^  a.  Similarly,  if  a  boolean  variable  ^  of  P 
does  not  appear  in  Q,  then  adjoin  0^  =  to  ©.  Note  that  neither  of  these  changes  affect  the 
read-once  condition  of  P  or  Q,  nor  the  single  occurrence  property  for  O  or  0. 

Let  %[a  <r-  Tia]  denote  for  each  boolean  variable  a  in  the  substitution  of  formula  71^. 
Now,  we  claim  that  the  composition  IT  °  0  =  {Qb[a  <r-  71^]}  expresses  P;  Q; . 

P; 


a 


b 

syntax  tree  for  Qb[a  <-  JCa] 

Moreover,  the  single  occurrence  property  for  0  guarantees  that  each  formula  Tt^  in  n  is 
used  exactly  once  to  replace  an  occurrence  of  the  boolean  variable  a  in  0.  Since  all  the 
original  boolean  variables  occurring  in  0  are  replaced  in  this  manner,  the  only  remaining 
boolean  variables  in  n  «  0  are  those  occurring  in  n.  Since  each  formula  tc^  in  n  was  used 
exactly  once,  and  since  O  satisfies  the  single  occurrence  property,  we  can  see  that  11  o  © 
satisfies  the  single  occurrence  property  too. 

Next,  consider  the  program  LOOP  h  P;  which  is  by  far  the  most  difficult  part  of  the 
argument.  By  induction  hypothesis,  there  is  a  collection  of  first-order  formulas  n(x)  = 
{7i^(x)}  with  the  single  occurrence  property,  such  that  the  value  of  b  after  executing  P  is 
nb[h],  for  h  equal  to  the  position  of  the  loop  variable  h  (other  unbound  head  positions 
have  been  ornitted  for  clarity). 

First  remove  negations  by  rewriting  P.  Push  all  negations  to  the  bottom,  then  replace 
every  occurrence  of  NOT  b  by  a  new  boolean  variable  Jb '.  Now,  follow  each  assignment 
b  :=  e;  by  h':=  NOT  e; ,  where  again  we  push  negations  to  the  bottom.  Since  every 
boolean  variable  appearing  in  e  occurs  once,  both  a  boolean  variable  and  its  negation  cannot 
both  be  in  e,  so  the  new  program  still  satisfies  the  read  once  condition. 

The  single  occurrence  property  insures  that  the  syntactic  dependency  graph  of  fl, 
defined  as  the  graph  whose  vertices  are  boolean  variables  in  IT,  and  whose  edges  are  given 
by  {(a,  b) :  a  appears  in  Tt^,}  for  all  boolean  variables  a  and  b,  has  out-degree  <  1.  This 
means  it  looks  like  a  bunch  of  disjoint  whirlpools. 


syntactic  dependency  graph 


Since  boolean  variables  in  disjoint  components  are  independent  of  each  other,  then  without 
loss  of  generality  it  suffices  by  syntactic  decomposition  to  consider  a  program  P  with  only  a 
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single  component.  We  now  confine  our  discussion  to  such  a  case,  and  define  the  order  of 
P  to  be  the  length  m  of  the  cycle  that  appears  (see  numbers  in  figure),  reserving  0  for  the 
case  when  there  is  a  root  instead  of  a  cycle.  And  define  the  height  of  P  to  be  the  length  h  of 
a  longest  path  from  any  node  to  its  closest  node  on  the  cycle  (or  root  if  there  is  none).  Vari¬ 
ables  on  the  cycle  are  termed  recursive  ^  and  variables  off  the  cycle  are  said  to  be  of  finite 
dependency. 

The  plan  is  to  break  up  the  n  iterations  of  the  loop  into  three  sections,  for  n  (equal  to  the 
length  of  the  input)  sufficiently  large.  The  first  section,  called  the  ‘leader’ ,  consists  of  an 
initial  run  of  h  iterations.  This  is  followed  by  the  ‘main  loop’ ,  which  repeats  an  m-iterate 
block  a  total  of  I- (n-h)  div  m  times.  This  in  turn  is  followed  by  a  ‘trailer’  of  the  remain¬ 
ing  (n  -  h)  mod  m  iterations. 

Leader 

As  long  as  «  >  /i,  the  composition 

A  =  n[0]°...on[/i-l] 

is  clearly  first-order  expressible.  Furthermore,  since  each  boolean  variable  b  of  finite 
dependency  has  height  less  than  or  equal  to  K  each  corresponding  A  will  have  no 
boolean  variables  occurring  in  it,  and  we  call  such  formulas  explicit.  Since  A  is  a  finite 
power  of  n  under  composition,  the  collection  A  =  including  the  recursive  boolean 

variables  b  still  satisfies  the  single  occurrence  property. 

Main  Loop 

We  now  proceed  with  the  important  task  of  contracting  the  cycle.  If  m  =  0,  there  is  no 
cycle  to  contract,  it  is  easy  to  see  that  the  dependency  graph  is  a  ^ee,  and  hence  each  boo¬ 
lean  variable  has  finite  dependency.  In  this  case,  each  boolean  variable  will  have  an  explicit 
formula  to  describe  its  value,  essentially  consisting  of  the  last  h  iterations  of  the  loop.  The 
details  can  be  determined  by  continuing  the  proof  for  the  case  m  >  0  and  just  skipping  the 
parts  which  deal  with  a  recursive  boolean  variable.  On  the  other  hand,  if  m  >  1,  then  we 
simplify  things  by  dividing  the  remaining  n-h  iterations  into  I  =  (n-h)  div  m  blocks  of 
size  m,  writing  each  such  block  as 

©[/]  =  n[/i  +  j-m]  °  ...  o  Tl[h  +  j  m  +  m-1]  for  7  =  0,  ...  / -  1 

It  needs  to  be  noted  that  starting  the  loop  variable  at  h  and  incrementing  it  in  multiples  of  m 
is  first-order  expressible  using  arithmetic,  and  that  the  dependency  graph  for  ©  satisfies  the 
single  occurrence  property  (being  a  fixed  power  of  11).  The  purpose  of  all  this  is  to  obtain  a 
dependency  graph  for  0  with  components  of  order  1,  so  that  each  recursive  boolean  vari¬ 
able  is  in  its  own  cycle. 


one  component  of  the  syntactic  dependency  graph  for  0 
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The  main  loop  consists  of  composing  ©[/]  from  ;  =  0  to  /  -  1.  Coming  into  the  main 
loop,  all  Xb  for  b  non-recursive  are  explicit  formulas  (from  the  leader).  We  shall  demon¬ 
strate  that  the  partial  iterates  defined  by 

Ao0[O]o©[l]o...  oe[k-l]  for  0<k<l 


can  be  expressed  by  a  collection  of  first-order  formulas  ^(jc)  =  {\1/^(jc)},  with  the  single 
occurrence  property.  Intuitively,  will  express  the  value  of  b  before  going  into  B[k] 

(after  h  +  hm  iterations  of  P)  and  the  final  value  of  b  after  completing  the  main  loop  will  be 
given  by  \|;^[/]. 

We  first  derive  the  formulas  \|/£,(x)  for  the  values  of  non-recursive  boolean  variables  b  at 
each  stage  of  the  main  loop,  by  induction  on  their  height  (the  longest  path  from  a  leaf). 
Let  0^(x)  be  the  first-order  formula  for  b  in  0(x). 

y^b(x)  <=>  (x  =  0)  AXb  V  (x  >  0)  A  0i,(x  -  l)[a  <-  Va(x  -  1)] 

If  ^  is  a  leaf  in  0,  then  0^,  contains  no  boolean  variables,  and  the  indicated  substitution  is 
vacuous.  However,  if  b  is  not  a  leaf,  then  the  replacements  xj/a  have  already  been  expressed 
by  induction  hypothesis  (each  a  occurring  in  0^,  is  by  definition  of  smaller  height).  Since 
Xb  contains  no  boolean  variables,  it  follows  by  induction  that  neither  does  \\fb,  since  as  long 
as  b  is  not  recursive,  each  a  is  not  either. 

The  recursive  variables  are  the  most  interesting  part.  Let  ^  be  a  recursive  boolean  vari¬ 
able,  and  let  0^(x)  e  0.  The  only  occurrence  of  a  recursive  boolean  variable  in  0^(x)  is  a 
single  positive  occurrence  of  b  itself,  by  virtue  of  the  fact  that  the  dependency  graph  for  0 
has  order  1,  and  because  we  have  eliminated  negations  while  preserving  the  read-once  con¬ 
dition.  In  particular,  this  makes  0^(x)  monotone  in  b.  So  b  is  true  at  the  end  of  the  main 
loop  just  in  case  there  is  a  last  stage  i  in  which  b  is  or  becomes  true  (here  is  where  we 
require  monotonicity)  and  remains  true  at  all  subsequent  stages  j  >  i.  In  explaining  this 
we’ll  use  T  to  stand  for  true,  1  for  false,  and  0^[i;  f]  to  stand  for  Bb[i][b  f-  t],  where  t  is 
a  boolean  formula.  For  b  to  be  or  become  true  at  stage  i  >  0  means  0£,[/;  _L],  or  in  the  case 
of  i  =  0, 0i,[O;  Xb],  where  b  enters  the  main  loop  with  value  Xb  that  it  obtains  from  the  lead¬ 
er,  t  To  make  things  simpler,  we  can  combine  these  two  cases  into  the  single  formula 
%[i‘,  i-OAXb].  For  b  to  remain  true  at  stage  j  >  i  means  Qb\j\  T],  which  can  be  included 
in  a  further  combination  as  0^[/;  j>ivj  =  0  a  Xb]  for  j  >  i.  Of  course,  the  values  of  the 
other  (non-recursive)  boolean  variables  a  that  0^,  may  depend  on  must  be  known  at  every 
stagey  of  the  main  loop  in  order  for  this  to  work.  But  recall  that  we  have  already  ascer¬ 
tained  these  formulas  \l/a[/],  so  we  can  substitute  those  occurrences  out.  The  value  of  ^  at 
the  end  of  the  main  loop  is  given  by: 

y^bU]  =  (3x.O  <x<  l)0fyjc  <y<  l)Bb(y)[b  <r-y>xvy  =  0  AXb;a<r-  \\fa(y)] 

where  a  ranges  over  all  (non-recursive)  boolean  variables  appearing  in  Qb-  To  help  under¬ 
stand  this,  the  following  picture  graphs  the  boolean  value  of  b  (shown  on  the  vertical  axis) 
through  time  (as  shown  by  j  on  the  horizontal  axis)  in  the  case  where  \\fb[l]  ends  up  true. 


t  It  doesn’t  matter  if  b  is  already  true  at  point  i,  since  monotonicity  insures  that  0fr[/;  1]  ^  Qb[i\  T]. 
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By  construction,  the  special  clause  j>  ivj  =  0  AXb  evaluates  to  the  correct  boolean  value 
in  all  cases.  To  see  that  the  resulting  collection  4^  satisfies  the  single  occurrence  property, 
note  that  for  each  non-recursive  boolean  variable,  \j/a  contains  no  boolean  variables,  as  we 
observed  earlier.  For  the  remaining  recursive  boolean  variables,  recall  that  the  collection 
A  =  satisfies  the  single  occurrence  property.  Now  observe  that  ¥  and  A  «  0  have  the 
same  dependency  graph  by  examination  of  the  constructed  formulas.  Since  both  A  and  0 
have  the  single  occurrence  property,  their  composition  must  also. 

Trailer 

The  remaining  iterations  (anywhere  from  0  to  m  - 1)  can  be  expressed  by 
T  =  n[n  -  ((n - /i)  mod  m)]°  ...°  n[n-  1] 


The  arithmetic  involved  is  certainly  first-order  definable  as  a  function  of  n,  which  also 
determines  the  length  of  the  composition.  However,  the  astute  reader  will  notice  that  we 
would  not  be  able  to  maintain  the  single  occurrence  property  for  T  if  we  were  to  just  com¬ 
bine  the  m  possible  cases  implied  above  into  a  single  formula.  But  over  all  n,  there  are  only 
m  possible  syntactic  dependency  graphs  for  T.  For  each  of  those  possibilities,  T  has  the 
single  occurrence  property  guaranteed  by  composition,  and  we  can  use  'P  «  T  to  express  the 
semantics  of  the  loop. 

Coda 

As  a  matter  of  fact,  we  see  from  the  trailer  that  the  end  result  is  parametrized  by  n  over 
finitely  many  choices.  So  technically  speaking ,  it  is  necessary  to  stipulate  from  the  begin¬ 
ning  that  each  program  P  is  represented  by  one  of  a  finite  number  of  collections  of  first- 
order  formulas  (each  individually  satisfying  the  single  occurrence  property).  The  proof  is 
then  complete  only  by  observing  that  this  technics  construction  can  be  carried  forward 
through  Ae  proof  without  affectation,  the  important  thing  being  that  the  number  of  possibil¬ 
ities  grows  with  each  application  of  the  induction  hypothesis,  but  remains  bounded  and 
first-order  dependent  only  on  n.  The  final  result  of  a  boolean  variable  in  a  program  is  then 
expressed  as  a  boolean  combination  of  these  finitely  many  cases. 

If  perchance  n  is  too  small  to  carry  out  the  whole  above  construction,  just  enumerate  the 
finitely  many  cases  involved. 

§  5  Directions 

Improvements 

In  the  model  we  have  defined,  the  use  of  a  binary  counter  to  monitor  absolute  head 
position  is  somewhat  inelegant,  and  the  looping  construct  to  bind  head  movement  is  some¬ 
what  restrictive.  It  would  be  much  nicer  if  the  heads  could  be  controlled  by  MOVE  instruc¬ 
tions,  and  loops  replaced  by  DO  h  TIMES  ...  for  some  head  position  h.  The  lack  of  con¬ 
ditionals  would  retain  obliviousness,  and  these  instructions  alone  would  be  sufficient  to 
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obtmn  the  necessary  arithmetic,  without  having  to  keep  track  of  relative  and  absolute  head 
positions.  Unfortunately,  this  may  compute  too  much,  and  the  success  of  this  more  aes¬ 
thetic  approach  seems  to  depend  on  resolving  a  certain  fundamental  question  in  finite  model 
theory,  namely:  does  iteration  (as  a  logical  construct)  over  finite  initial  segments  of  the  nat¬ 
ural  numbers  with  successor  close  at  FO(+,  *)?  Surprisingly,  iterated  multiplication 
(exponentiation,  is  in  FO(+,  *).  However,  a  more  promising  approach  which  doesn't 
rely  on  complexity  theoretic  separations  might  be  to  just  devise  some  simple  mechanism  of 
tying  head  movements  to  a  global  system  clock  using,  for  example,  frequency  dividers. 


Extensions 

Other  Complexity  Classes 

It  is  relatively  easy  to  obtain  a  sequential  deterministic  characterization  of  ALOGTIME 
(uniform-NCl)  by  dropping  the  destructive  read  restriction  in  our  constant-space  model  (in 
fact  the  proof  would  be  much  easier,  cf.  [BI]).  This  identifies  ALOGTIME  in  a  reasonably 
elegant  manner  with  constant-space  serial  algorithms.  To  my  mind  the  real  challenge  is  to 
find  a  similar  sequential  deterministic  model  for  uniform  constant-depth  threshold  circuits 
(TC®).  Presumably,  the  model  might  use  integer  variables  vrith  the  read-once  restriction.  It 
would  also  be  instructive  to  directly  prove  the  containment  TC^  c  NCl  in  this  setting, 
demonstrating  a  concrete  constant-space  serial  algorithm  for  counting. 

Time/Space  Duality 

We  observed  in  section  3  that  the  standard  schoolbook  algorithm  for  binary  addition  is  a 
read-once  constant-space  serial  algorithm.  If  you  go  through  the  proof  of  our  main  theo¬ 
rem,  you  will  obtain  a  first-order  formula  for  it  which  can  be  seen  to  be  virtually  the  same 
as  the  classic  cany  look-ahead  method: 

s{i)  =  a{i)  0  b{i)  0  c{i)  where, 

c(i)  =  (3j)[l  <  i  A  a(i)  a  b(i)  a  (Vk)[/  <k<i~^  (a(i)  v  ^(i))]] 

Perhaps  the  best-known  (and  probably  oldest)  parallel  algorithm,  it  says  that  there  is  a  carry 
into  a  column  if  and  only  if  some  previous  column  generated  one,  and  every  subsequent 
column  propagated  it. 

A  somewhat  surprising  consequence  of  our  work  is  that  in  general,  for  polynomial 
bounds  on  size  and  length,  destructive  read  constant-space  serial  algorithms  are  equivalent 
to  constant-time  parallel  algorithms.  Classical  parallel-time  /  serial-space  duality  is  a  phe¬ 
nomenon  that  appears  only  to  extend  down  to  resource  bounds  of  0(log  n)  [H].  However, 
the  seemingly  peculiar  read-once  restriction  has  allowed  us  to  take  this  duality  all  the  way 
down  to  0(1)  time  or  space.  By  modifying  our  model  to  allow  dynamically  allocated  new 
storage  (additional  boolean  variables),  it  should  be  possible  to  use  Immerman’s  iterated 
first-order  formulas  to  extend  our  results  to  provide  a  tight  correspondence  between  read- 
once  space  and  quantifier-depth.  One  particularly  intriguing  possibility  is  a  duality  theorem 
for  TC^,  involving  the  as  yet  undiscovered  sequential  model  mentioned  above. 

Multiplication 

Since  n-bit  multiplication  is  in  TC®,  it  is  instructive  to  consider  the  usual  schoolbook 
algorithm  for  binary  multiplication  as  an  example  of  a  serial  algorithm  with  integer  varia¬ 
bles. 
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dn-l 
X  Sn-i 


do 

So 


=  P2n-1  . PO 

The  partial  products  (not  pre-computed)  are  added  right  to  left  in  columns,  and  the  par¬ 
tial  sums  for  each  column  accumulated  top  to  bottom,  with  a  multi-bit  carry  into  the  next 
column. 


s  :=  0 

LOOP  i  FROM  0  TO  2*22-1 
LOOP  j  FROM  0  TO  i 

s  :=  s  +  d{j)  *  e(i-j) 
p(i)  :=  s  mod  2 
s  :=  s  div  2 


{initialize  partial  sum) 
{go  from  LSB  to  MSB} 
{Siam  for  ith  column} 
{add  next  term  in  Siam} 
{product  bit} 
{carry  to  next  column} 


Notice  that  this  uses  variable  loop  bounds  and  the  read/write  integer  variable  s  of  OQog  n) 
bits.  Even  though  s  appears  to  be  read  twice,  the  operations  of  incrementing  and  halving,  if 
combined,  are  read-once  at  the  bit  level.  It  is  intriguing  to  wonder  if  a  parallel  dual  to  this 
serial  algorithm  might  not  provide  a  simplified  witness  to  the  fact  that  multiplication  can  be 
performed  in  uniform-TCO.  Unlike  carry  look-ahead,  the  current  parallel  constructions  are 
rather  more  complex  and  involved  than  this  simple  and  intuitive  serial  method  [JL]. 
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Abstract.  We  introduce  the  notion  of  a  logic  capturing  a  relativized 
complexity  class  uniformly  by  treating  both  generalized  quantifiers  and 
oracles  as  indeterminates  and  requiring  that  the  correspondence  be  uni¬ 
form.  Besides  reinterpreting  previous  results  from  this  point  of  view,  we 
show  that  Fixed  Point  Logic  with  inflationary  fixed  points  I  FPL  cap¬ 
tures  P  uniformly,  whereas  First  Order  Logic  FOL[Kiy . . .  yKm]  with 
any  number  of  generalized  quantifiers  cannot  capture  uniformly  a  rel¬ 
ativized  complexity  class  which  contains  NC2.  This  contrasts  the  fact 
that  in  the  non-uniform  approach  both  I  FPL  and  FOL[ATC]  capture 
P. 

1  Introduction  and  Survey 

In  this  paper  we  continue  our  investigations,  cf.  [MP93,  MP94],  of  the  relation¬ 
ship  between  logics  augmented  with  generalized  quantifiers  and  oracle  computa¬ 
tions.  Intuitively,  we  think  of  a  logic  as  a  descriptive  language  and  of  generalized 
quantifiers  as  library  calls.  The  underlying  logic  allows  us  to  compose  such  li¬ 
brary  calls  into  more  complex  programs.  Similarly,  we  can  think  of  Oracle  Turing 
machines  as  computing  devices  with  oracle  calls.  Our  purpose  is  to  match  logics 
augmented  with  generalized  quantifiers  with  classes  of  Oracle  Turing  machines 
(Relativized  Complexity  Classes)  in  a  uniform  way. 

This  gives  us  not  only  a  reinterpretation  of  previous  results,  but  also  a  deeper 
insight  into  the  mechanism  of  how  logics  capture  complexity  classes.  It  also 
allows  us  to  distinguish  the  power  of  the  underlying  logic  from  the  power  of 
particularly  chosen  generalized  quantifiers.  When  this  distinction  is  not  made, 
every  logic  can  be  viewed  as  being  First  Order  Logic  augmented  by  a  set  of 
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a  Grant  of  the  German-Israeli  Foundation  (1995)  and  by  the  Fund  for  Promotion  of 
Research  of  the  Technion-Israeli  Institute  of  Technology. 

**  Minerva  Fellow  1993-1994. 
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generalized  quantifiers.  In  particular,  First  order  Logic,  Fixed  Point  Logics  and 
Second  Order  Logic  appear  on  the  same  level,  cf.  [MP93].  For  example,  the 
complexity  class  P  (Polynomial  Time)  is  captured  by  FOL[ATC]  (First  Order 
Logic  with  Alternating  Transitive  Closure),  [Imm87]  and  also  by  Af FPL  (Fixed 
Point  Logic  with  Monotone  Fixed  Points),  [Var82].  The  logic  IF  PL  (Fixed  Point 
Logic  with  Inflationary  Fixed  Points)  has  the  same  expressive  power  cis  MFPL^ 
as  shown  in  [GS86,  LeiQO].  By  introducing  our  distinction,  we  can  exhibit  the 
intrinsic  difference  between  these  two  approaches:  FOL[ATC,  Q]  will  be  different 
from  /FPL[Q],  when  Q  is  a  generalized  quantifier  treated  as  a  variable.  Once 
spelled  out,  this  is  not  surprising,  but  illuminating. 


Outline  of  the  paper 


We  assume  the  reader  is  familiar  with  the  basics  of  generalized  quantifiers  and 
logics  as  described  in  [EFT80,  MP93,  BF85]  and  with  logics  capturing  complexity 
classes,  as  in  [Imm87,  Imm88,  Imm89]. 

In  section  2  we  introduce  our  notion  of  uniformly  capturing  of  relativized 
complexity  classes. 

In  this  section  we  give  also  a  detailed  outline  of  our  results.  In  section  3 
we  discuss  various  interpretations  of  our  results  and  their  impact  on  complexity 
theory.  In  this  section  we  also  explain  and  correct  an  imprecision  in  [MP94]. 

In  section  4  we  give  the  exact  definitions  of  our  uniform  approach  to  model 
checking  and  introduce  the  query  dependency.  This  is  used  to  formulate  and 
prove  our  main  technical  lemma  32.  It  is  also  used  to  prove  upper  bounds  on 
uniform  model  checkers. 

In  section  5  we  define  a  relativized  (or  oracle)  language,  Lo(F),  which  is  a 
modified  version  of  an  oracle  language  used  by  J.  Buss  [Bus88,  Bus86]. 

In  section  6  we  determine  the  exact  complexity  of  Lq{K). 

In  section  7  we  use  this  oracle  language  to  give  a  proof  of  our  main  theorems 
11,  31,  34. 

In  section  8  we  draw  our  conclusions  and  formulate  some  conjectures. 
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2  Logics  Capturing  a  Relativized  Complexity  Class 
Uniformly 

In  this  section  we  introduce  our  notion  of  logics  capturing  a  relativized  complex¬ 
ity  class  uniformly.  We  view  formulas  of  logics  with  generalized  quantifiers  as 
type-2  objects  (depending  on  the  interpretation  of  the  quantifiers).  Similarly, 
we  view  oracle  languages  as  type-2  objects  (depending  on  the  choice  of  the  con¬ 
crete  oracle).  We  call  the  latter  both  relativized  and  oracle  complexity  classes, 
depending  the  aspect  we  wish  to  stress. 


2.1  Oracle  Turing  Machines  and  Languages 

Definition!  ([GJ79]).  An  Oracle  using  Turing  Machine  (OTM)  is  a  tuple 
{Q,r,8^  where  Q  is  a  finite  set  of  states,  F  is  the  tape  alphabet  and  5  is  a 
transition  function. 

From  this  definition  it  is  clear  that  the  description  (as  opposed  to  the  behaviour) 
of  an  OTM  is  independent  of  the  specific  oracle  set  used.  We  make  this  distinction 
explicit  in  the  following 

Definition  2  (Oracle  Languages).  Let  D  be  a  class  of  Turing  machines  with 
some  time  and  space  bounds  (a  complexity  class). 

(i)  OTM{fD{K)),  the  set  of  Oracle  Turing  Machines  with  oracle  parameter  K, 
which  are  in  D. 

(ii)  L{M{K)),  for  M{K)  €  OTM{fD{K)),  the  language  accepted  by  M{K). 

(iii)  D^,  the  set  of  languages  accepted  by  Oracle  Turing  Machines  in 
OTM{l^{K)). 

(iv)  We  denote  by  Lang[A)  the  set  of  languages  over  the  alphabet  A.  Let  L  : 
Lang{A)  — >•  Lang{A)  a  function  and  K  G  Lang{A).  We  denote  by  L{K) 
the  languages  obtained  by  applying  L  to  K.  The  family  of  languages  L{K) 
obtained  uniformly  using  L  are  called  oracle  languages. 

(v)  An  oracle  language  L{K)  is  said  to  be  uniformly  recognized  by  M  E 
OTM(D{K)  if  for  every  language  K  M(K)  recognizes  L{K). 

Definitions  (Uniformly  Capturing).  Let  C[Q]  be  a  logic  with  a  distin¬ 
guished  generalized  quantifier  and  D^  an  oracle  complexity  class.  We  say  that 
C[Q]  captures  D^  uniformly  if  there  are  functions 

(i)  tri  :  C[Q]  OTM{fD{K))  such  that  for  every  oracle  K  and  for  every 
formula  <j)  G  C{Q]  the  translation  of  <j),  tri((f),  is  a  model  checker  for  <j>{QK)\ 
and 

(ii)  tr2  :  OTM{'D{K))  — >•  C[Q]  such  that  for  every  oracle  K  and  for  every  oracle 
Turing  machine  M{K)  G  OTM{p{K)),  the  models  of  the  translation  of 
M{K)  are  exactly  the  strings  accepted  by  Af(A'),  i.e.  Mod{tr2{M{K)))  = 
L{M{K)). 
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2.2  Positive  Results 

In  [MP94]  we  have  shown  the  following: 

Proposition4  (Makowsky-Pnueli).  We  denote  by  u{oo)  the  restriction  of 
Buss*  unbounded  oracle  computation  models  [Bus88]  to  bounded  stacks,  as  intro¬ 
duced  in  [MP94]^ 

(i)  ForL^  with  the  oracle  model  u(oo)  FOL[DTC,K]  captures  uniformly. 

(ii)  For'NL^  with  the  oracle  model  u{oo)  FOL[TC,K]  captures  NL^  uniformly. 
(Hi)  For  AL^  with  the  oracle  model  t/(oo)  FOL[ATC,K]  captures  AL^  uni¬ 
formly. 

Although  AL  =  P,  we  shall  see  below  that  for  the  u{oo)  oracle  computation 
model  there  are  oracles  K  such  that  AL^  may  be  different  from  P^. 

To  capture  P^  uniformly  we  look  first  at  Fixed  Point  Logic  with  inflationary 
fixed  points  {I FPL).  This  choice  allows  the  introduction  of  generalized  quanti¬ 
fiers  without  technical  problems.  It  is  rather  straight  forward  to  see  that,  using 
the  standard  oracle  consultation  model  of  Ladner  and  Lynch,  [LL76],  we  have 
the  following: 

Theorems.  For  P^  with  the  standard  oracle  model,  IFPL[K]  captures  P^ 
uniformly. 

Proof  We  have  to  write  down  the  behaviour  of  the  OTM  in  IF  PL.  The  proof 
follows  closely  the  presentation  in  [AHV94]  or  [EF95].  The  main  difference  is, 
that  in  the  description  of  the  transition  table  of  the  Turing  Machine  we  need 
the  general  quantifier  for  the  oracle  consultation  and  the  formulas  in  the  scope 
of  the  general  quantifier  contain  further  application  of  the  fixed  point  operator 
for  decoding  the  contents  of  the  oracle  tape. 

We  could  also  look  at  the  case  of  Fixed  Point  Logic  where  the  variable  of  the 
fixed  point  operator  occurs  only  positively  in  the  formula  (MFPL).  In  the  absence 
of  generalized  quantifiers  this  garantees  monotonicity.  IF  PL  and  MFPL  both 
capture  P  and  their  intertranslat ability  was  shown  in  [GS86]  and  simplified  in 
[Lei90].  The  notion  of  capturing  P  uniformly  is  not  well  defined  for  MFPL. 
We  have  to  restrict  ourselves  to  oracles  K,  for  which  we  know  that  positive 
occurrence  of  the  variables  still  ensures  monotonicity.  We  call  such  oracles  K 
monotone. 

With  this  definition  it  is  easy  to  show: 

Theorems,  For  P^  with  the  standard  (unbounded)  oracle  model,  MFPL[K] 
captures  P^  uniformly  for  monotone  oracles. 

However,  it  is  not  clear  whether  the  intertranslatability  of  IFPL[K]  and 
MFPL[K]  holds  uniformly  for  monotone  K. 

Looking  at  Fixed  Point  Logic  in  an  unrestricted  way  (PFPL)  the  choice 
of  the  oracle  consultation  model  is  again  crucial.  G.  Gottlob  has  pointed  out 
that  for  the  oracle  computation  model  introduced  by  I.  Simon,  [Sim77],  which 
is  equivalent  to  the  bounded  model  of  Buss,  [Bus88],  one  can  show: 
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Theorem?  (Gottlob).  For  PSpace^  with  the  bounded  oracle  model, 
PFPL[K]  captures  PSpace^  uniformly. 

Remark.  If  we  use  SOL,  Second  Order  Logic,  rather  than  Fixed  Point  Logic,  we 
get  similar  results  for  the  polynomial  hierarchy  PH,  [MP95]. 

Remark.  A  Theorem  similar  to  Theorem  6  was  also  noticed  by  1.  Stewart  in 
[Ste94].  Stewart  showed  that  MFPL[HAM],  monotone  fixed  point  logic  aug¬ 
mented  with  the  quantifier  expressing  Hamiltonicity,  captures  P^^.  However,  he 
does  not  address  the  issue  of  replacing  HAM  by  other  oracles,  and  therefore  the 
question  of  uniformity  does  not  arise.  Also  G.  Gottlob,  [Got95a,  Got95b],  deals 
with  related  questions  concerning  logics  capturing  relativized  logspace  complex¬ 
ity  classes,  where  the  base  logic  is  First  Order  Logic  FOL.  It  is  clear,  that  FOL, 
in  contrast  to  FOL[DTC]  (4),  cannot  capture  L  uniformly.  We  return  to  the 
question  of  what  FOL  could  capture  uniformly  in  section  8. 


2.3  Negative  Results 

Now  we  discuss  negative  results.  The  first  is  an  observation  of  G.  Gottlob  con¬ 
cerning  the  choice  of  oracle  consultation  models  in  the  case  of  P  Space.  In  con¬ 
trast  to  theorem  7  with  the  bounded  oracle  consultation  model,  under  the  stan¬ 
dard  oracle  consultation  model  we  have: 

Propositions  (Gottlob). 

(i)  NEXPTime  C  PSpace'^*’,  but 

(it)  Every  formula  of  PFPL[K]  has  a  model  checker  in  PSpace  for  oracles 
K  €  NP. 

Proof,  (i)  NEXPTime  C  P Space‘s P:  This  inclusion  is  due  to  the  fact  that  un¬ 
der  the  standard  oracle  model,  i.e.,  the  Ladner- Lynch  model,  the  oracle  string  is 
not  subject  to  any  size-bound.  This  means  that  the  PSpace  machine  may  write 
exponential  strings  to  its  oracle-tape.  The  oracle  is  NP,  but  with  exponential 
input  it  behaves  like  NEXPTime  w.r.t.  the  overall  OTM  input  string.  Thus, 
any  problem  in  NEXPTime  can  be  solved  by  a  PSpace^P  machine  under  the 
standard  oracle  model. 

(ii):  Consider  partial  hxpoint  logic  PFPL.  Assume  PFPL  is  enriched  by  a 
generalized  quantifier  Qk  corresponding  to  some  oracle  K  €  NP.  Then,  given 
the  fixed  arities  of  the  signature,  any  formula  <^(x)  G  PFPL[K],  where  x  denotes 
a  list  of  free  variables,  evaluates  to  a  polynomially  sized  relation  for  each  finite 
structure  A.  Thus  any  formula  Qk^<I>{^)  corresponds  to  an  application  of  the 
quantifier  Qk  to  a  polynomial  relation,  or,  equivalently  to  an  oracle-call  of  oracle 
K  fed  with  a  polynomial  size  string.  Thus,  in  PFPL[K],  there  is  no  way  of 
feeding  an  NP-oracle  with  an  exponential  string,  and  actually,  since  K  6  NP, 
it  holds  that,  by  abuse  of  notation,  PFPL[K]  =  PFPL  =  PSPACE. 

Thus  we  get,  in  contrast  to  theorem .7 
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Theorem  9  (Gottlob).  Assume  NEXPTime  /  PSpace.  Then  under  ihe 
standard  oracle  consultation  model  of  Ladner  and  Lynch  PFPL  cannot  capture 
PSpace  uniformly. 

To  understand  better  the  difference  between  logics  uniformly  capturing  a 
complexity  class,  and  the  usual  notion  of  logics  capturing  a  complexity  class,  let 
us  look  at  the  following 

Proposition  10  (Immerman).  If  Ki  is  D -complete  for  first  order  reductions, 
then  FOL[Ki]  captures  D  in  ihe  following  way:  Let  Kq  £  D  and  lei  be  ihe 
first  order  reduction  from  Kq  to  Ki.  Then  the  formula  Qki^i  defines  Kq. 

However,  if  we  replace  Ki  by  some  K2  which  is  also  D-complete  for  first 
order  reductions,  the  formula,  which  defines  Kq  is  not  the  formula  Qk^^i  *  To 
define  Kq,  now,  we  need  a  first  order  reduction  ^2  from  Kq  to  K2  and  the  formula 
Qk‘2^2  then  defines  Kq. 

From  this  point  of  view  a  logic  C  captures  a  complexity  class  D  uniformly  if 
it  can  simulate  the  Oracle  Turing  Machines  over  D.  The  oracle  Turing  Machines 
are  here  viewed  as  machines  pasting  together  oracle  calls  (or  library  programs) 
without  reference  to  their  contents.  So  the  logic  needs  a  certain  built  in  mecha¬ 
nism  to  simulate  D  modularly. 

Our  main  result  in  this  paper  is  a  negative  result.  It  involves  the  relativized 
classes  NC,-^  introduced  in  [Wil86]. 

Theorem  11.  Let  Ki, . . . ,  Km  he  fixed  oracles  (languages).  There  is  an  oracle 
language  L(K)  such  that 

(i)  L{K)  is  uniformly  recognizable  by  a  uniform  family  of  relativized  (oracle) 
circuits  with  size  0(n*),  for  some  i  G  IN  and  depth  0({logn)^). 

In  other  words  L{K)  G  NC2^  uniformly,  and  hence  L(K)  G  P^  uniformly. 

(ii)  L(K)  is  not  uniformly  definable  by  any  formula  <f>  G  FOL[Ki, . . . ,  Km,K]. 

3  Interpretation 

3.1  First  Order  vs  Fragments  of  Second  Order  Logic 

Theorems  5  (7)  and  11  spell  out  a  difference  over  finite  structures  between  First 
Order  Logic  with  generalized  quantifiers  and  Fixed  Point  Logic  with  generalized 
quantifiers.  The  former  is  only  capable  to  capture  uniformly  complexity  classes 
which  use  logarithmic  space  and  have  a  bounded  oracle  stack.  In  the  case  of 
non-deterministic  and  alternating  logarithmic  space  additional  attention  has  to 
be  given  to  the  exact  oracle  computation  model.  In  contrast,  the  Fixed  Point 
Logics  I  FPL,  PFPL  capture  uniformly  polynomial  time  (in  the  Ladner-Lynch 
model)  and  space  (in  the  Simon  model),  respectively. 

Although  from  a  model  theoretic  point  of  view,  a  logic  whose  expressive 
power  lies  between  First  Order  Logic  and  Second  Order  Logic  cannot  be  classified 
more  or  less  first  or  second  order.  However,  from  a  computational  point  of  view 
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our  analysis  of  generalized  quantifiers  via  the  notion  of  uniformly  capturing  of 
complexity  classes  allows  such  a  distinction.  FOL[ATC]  has  more  the  character 
of  an  extension  of  First  Order  Logic,  whereas  IF  PL  is  more  a  fragment  of  Second 
Order  Logic.  Both  capture  P,  but  IF  PL  does  so  uniformly,  and  therefore  reflects 
better  upon  the  inherent  computational  power  of  P. 

It  is  an  interesting  question,  which  we  do  not  pursue  further  in  this  paper, 
whether  First  Order  Logic  can  capture  uniformly  the  oracle  classes  associated 
with  ACq, 


3.2  Turing  Reductions 

There  is  another  way  of  looking  at  the  difference  between  FOL[ATC]  and  IF  PL, 
which  was  suggested  to  us  by  A.  Dawar.  On  ordered  finite  structures  FOL[ATC] 
captures  P  and  also  many-one  P-reductions  (Karp  reductions),  as  shown  inde¬ 
pendently  in  [Daw93,  Daw94]  and  in  [MP93,  MP94].  What  our  result  here  shows, 
is  that  FOL[ATC]  does  not  capture  Turing  reductions  (Cook  reductions).  The 
difference  is,  that  in  many-one  reductions  the  oracle  is  applied  only  once,  whereas 
in  Turing  reductions  it  may  be  applied  several  times.  What  FOL[ATC]  is  miss¬ 
ing  is  the  means  of  propagating  the  dependency  of  the  next  oracle  query  on 
the  result  of  a  large  number  of  previous  queries.  In  contrast  to  this  IF  PL  does 
capture  Turing  reductions. 

This  distinction  is  important,  because  work  in  complexity  theory  has  shown 
that  studying  reductions  allows  us  to  make  much  finer  distinctions.  For  instance, 
it  is  a  direct  consequence  of  the  Baker,  Gill  and  Solovay  result  ([BGS75]  that 
there  is  an  oracle  K  such  that  P^  is  different  from  NP^ ,  that  Turing  reductions 
in  P  are  strictly  weaker  than  Turing  reductions  in  NP.  Similarly,  in  this  paper 
we  show  that  while  FOL[ATC]  and  ILFP  both  capture  computations  in  P, 
only  the  latter  captures  reductions  in  P.  This  separation  of  the  two  logics  is 
analogous  to  the  separation  of  complexity  classes  in  a  relativized  world. 


3.3  A  correction 

Theorem  11  contradicts  a  claim  made  in  [MP94].  The  reason  behind  this  is, 
that  we  overlooked  an  additional  feature  of  Buss’  oracle  model  for  alternating 
logarithmic  space:  In  his  model  of  relativized  AL  it  is  possible  for  different 
branches  of  the  computation  tree  to  write  information  on  the  same  oracle  tape 
in  parallel,  a  feature  which  FOL  cannot  accommodate.  The  model  we  defined 
omits  this  mechanism.  For  Buss’  model  we  still  have  that  AL^^  =  AL“^  =  P^ 
for  every  K,  In  the  model  we  use,  there  are  Kq  and  a  language  L{Ko) 

such  that 

(i)  L(Ko)  e  NC2^°  C  P^o  but 

(ii)  L{Ko)  is  not  definable  uniformly  in  FOL[ATC,  Kq]. 

Now,  as  FOL[ATC,  K]  captures  uniformly,  we  get  additionally  that 
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Proposition  12.  There  exists  an  oracle  K,  such  that  ^  and 

hence  P^. 

For  the  oracle  computation  models  of  Ladner  and  Lynch  ([LL76],  Simon 
([Sim77])  and  Ruzzo,  Simon  and  Tompa  ([RST84])  it  was  known  that  oracles 
K  exist  with  AL"^  /  P^.  As  a  matter  of  fact,  J.  Buss  introduced  his  oracle 
consultation  model  to  fix  these  anomalies.  The  oracle  we  use  in  the  proof  of 
theorem  11  is  a  slight  modification  of  the  oracle  used  in  [Bus88]. 

4  Standard  Model  Checkers  (SMC) 

4.1  Model  Checking 

Let  Ki . .  .Km  be  a  set  of  sets  of  structures  and  let  Qi . .  .Qm  be  their  respec¬ 
tive  Lindstrom  quantifiers.  Let  S  =  FOL[Qi . .  .Qm]i  ^  —  FPL[Qi . .  .Qm]  and 
iS  SOL[Qi . .  .Qm]*  For  T  —  and  their  sublogics,  we  define  inductively 

standard  model  checkers  for  all  formulcis  6  jC  as  follows: 

Definition  13  (Assignments).  For  formulas  with  free  variables  the  input  of  a 
standard  model  checker  is  a  structure  and  an  assignment  for  the  free  variables. 

(i)  For  first  order  variables,  the  assignment  z{x)  gives  as  a  value  an  element  of 
the  structure. 

(ii)  For  second  order  variables  of  arity  k  z{U)  gives  as  a  value  a  subset  of  the 
A;“fold  cartesian  product  of  the  structure. 

(iii)  For  formulas  without  free  variables  the  input  of  a  standard  model  checker  is 
just  a  structure. 

The  output  is  always  a  yes  or  a  no. 

Definition  14  (SMC),  (i)  If  is  T  (F)  the  SMC{(1>)  simply  returns  yes  (no). 

(ii)  If  is  of  the  form  i2(a:i . . .  ajn)  then  the  SMC{(j))  checks  \=  (j>  in  some 

standard  way. 

(iii)  If  (j)  is  of  the  form  V’l  A  V’2,  (V’l  V  V’2,  “'V’)  SMC^j))  calls  the  model  checkers 
of  and  ^2  and  accepts  if  both  accept  (if  one  of  them  accepts,  if  5'MC'(V’) 
rejects). 

(iv)  If  <j>  is  of  the  form  '^xil){x)  (Va?V’(®))  fben  for  each  possible  substitution  of 
X  the  model  checker  of  \l){x)  is  invoked  with  z  modified  by  this  substitution 
for  X.  SMC{<I>)  accepts  if  one  (all)  these  model  checkings  accept. 

Note  that  if  the  structure  has  cardinality  n,  then  this  part  may  be  executed 
n-times,  but  these  checks  do  not  depend  on  each  other. 

(v)  If  (j)  is  of  the  form  3Ui){U)  then  for  each  possible  substitution  of 

U  the  model  checker  of  ip{U)  is  invoked  with  z  modified  by  this  substitution 
for  U.  SMC{<I>)  accepts  if  one  (all)  these  model  checkings  accept. 

Note  that  if  the  structure  has  cardinality  n,  then  this  part  may  be  executed 
2^”-times,  but  these  checks  do  not  depend  on  each  other. 
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(vi)  If  (j)  is  of  the  form  fiUxlj{U)  then  the  .model  checker  computes  the  inflationary 
fixed  point  Sd  of  and  then  checks  ^((7)  for  z{U)  =  5^. 

Note  that  if  the  structure  has  cardinality  n,  then  d  =  d{n)  is  a  polynomial 
in  n.  However,  here  each  iteration  depends  on  the  outcome  of  the  previous 
stage. 

(vii)  If  (f)  is  of  the  form  Qi^  then  SMC{<I>)  has  one  oracle  tape  using  Ki  for  each 
occurrence  of  Q,-.  On  it  it  writes  the  structure  ^{A)  and  answers  yes  if  the 
oracle  accepts  and  no  otherwise. 

Recall  that  ^{A)  is  the  structure  obtained  from  A  by  interpreting  the  for¬ 
mulas  of  ^  in  A.  To  obtain  A  we  need  to  invoke  the  SMCs  of  the  various 
subformulas  of 

Theorem  15. 

(i)  Let  <l>  E  E.  Then  there  is  a  OTM  using  logarithmic  space  such  that  for 
every  choice  of  oracles  Ki, . . Km)  the  machine  is  a  SMC  for  (j)  where 
the  quantifiers  are  interpreted  by  Ki,,  Km> 

(ii)  Let  ET,  Then  there  is  a  polynomial  time  OTM  Mtj,  such  that  for  every 
choice  of  oracles  Ki, .  ..^Km  M^f,  is  a  SMC  for  <!>  where  the  quantifiers  are 
interpreted  by  Ki,. Km- 

A  fortiori,  this  is  also  true  for  <l>  E  E. 

(Hi)  Let  <j)  E  S,  Then  there  is  a  exponential  time  OTM  Mff,  such  that  for  every 
choice  of  oracles  Ki, .  ..,Km  M^  is  a  SMC  for  </>  where  the  quantifiers  are 
interpreted  by  Ki,. . Km- 

(iv)  Actually,  for  (j>  E  S  the  OTM  uses  polynomial  space  and  its  time  limitations 
are  within  the  polynomial  hierarchy  PH. 

Proof  Easy,  from  the  description  of  the  standard  model  checker. 

4.2  Oracle  Interdependence 

Definition  16  (Q-rank).  For  formulas  of  E,T  and  S  we  define  the  Q-rank  as 
the  number  of  nestings  of  the  generalized  quantifiers. 

The  next  lemma  spells  out  the  crucial  property  which  makes  E  = 
FOL[Qi . .  .Qm]  different  from  F  —  FPL[Qi .  ..Qm]-  It  has  to  do  with  the  in¬ 
terdependence  of  oracle  consultations.  In  the  case  of  E  we  can  layer  all  possible 
oracle  consultations  into  finitely  many  layers,  say  a  number  which  depends 
only  on  (j).  Already  in  F,  the  number  of  layers  will  depend  on  the  structure  A. 
Let  us  make  this  precise. 

Definition  17.  Let  A  hen  r-structure  and  (j)  E  E{t).  We  define  inductively  sets 
of  structures  parametrically  definable  in  as  follows: 

(i)  For  every  subformula  ip  =  Qx^(x^y)  of  (j)  let  I^^q,(A)  consist  of  all  the 
structures  B  —  0{x,d)(A)-  Here,  ^(jc,d)(.4)  denotes  the  structure  defined 
by  0  in  A- 

Note  that  in  the  case  of  SOL  y  and  a  can  contain  second  order  variables 
and  relation  symbols,  respectively. 
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(ii)  consists  of  the  union  of  all  the  ,  where  V’  is  a  subformula  of 

<j)  of  Q-rank  j  +  I, 

(iii)  /<^(v4)  consists  of  the  union  of  all  the  ,  where  tp  is  any  subformula 

of  (j). 

Lemma  18  (Query  Dependency  I). 

(i)  For  every  formula  <j)  G  FOL[Qi, . . . ,  Qm]  there  is  a  polynomial  P  (in  the  size 
of  A)  such  that  for  every  structure  A  of  size  n,  contains  at  most  P{n) 
elements. 

(ii)  For  every  formula  (p  G  SOL[Qi,. . . ,  Qm]  there  is  a  polynomial  P  (in  the  size 
of  A)  such  that  for  every  structure  A  of  size  n,  contains  at  most  2^^^^ 
elements. 

(iii)  Let  (j>  G  SOL[Qi, . .  .,Qm]  and  xp  he  a  subformula  of<p  ofQ-rankj-\-l.  Then 
the  evaluation  of  SMC((p)  at  stage  SMC(tp)  depends  only  on 

In  other  words,  the  interdependency  of  oracle  consultations  is  of  depth 
hounded  hy  the  Q-rank  of  (p. 

Proof.  Easy,  from  the  definitions. 

Remark.  For  <p  G  MFPL[Qi, . . . ,  Qm]  the  lemma  is  not  true.  In  the  evaluation 
of  the  fixed  point,  intermediate  relations  have  to  be  computed  which  do  depend 
on  each  other. 

Lemma  19  (Query  Dependency  II),  Let  (p  G  FOL[Qi, . . .  ,Qm]  he  given. 
Then  there  is  a  constant  k^  and  a  polynomial  P{n)  <  such  that  for  every 
choice  of  oracles  Ki, . . .,  Km  ond  for  every  structure  A  and  every  assignment  z 
SMC{<p,A,  z)  asks  at  most  P(n)  many  queries. 

Proof.  Easy,  from  the  definitions  and  the  previous  lemma. 

We  use  this  lemma  to  justify  the  following 

Definition20  (Diagonalization  parameter).  We  denote,  for  <p  as  above,  by 
i'(<p)  the  smallest  number  n  such  that 

<  2*^. 

i'{(p)  is  called  the  diagonalization  parameter  of  <p. 

5  The  Language  Lo{K) 

Here  we  consider  binary  languages  (subsets  of  {0, 1}*)  and  use  them  both  as 
oracles  and  as  sets  of  structures.  We  use  elements  of  {0, 1}*  also  to  denote  natural 
numbers  in  binary  notation.  For  x  G  {0, 1}*  we  denote  by  l{x)  the  length  of  x. 

Given  an  oracle  K,  we  define  our  language  Lq(K)  ,  which  follows  an  idea  of 
J.  Buss  [Bus88],  section  4.1.  He  uses  his  language  to  separate  NL^  from  P^. 
Our  definitions  below  are  basically  a  special  case  of  his. 

First  we  associate  with  an  oracle  K  (a  set  of  strings)  an  infinite  string  S{K): 
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Definition  21.  For  an  arbitrary  oracle  K,  let  S{K)  be  the  following  infinite 
binary  string:  si  -  the  first  bit  of  S{K)  is  1  if  ”0”  €  K  and  0  otherwise.  5*  -  the 
rth  bit  -  is  1  if  Si , .  .Sj_i  G  K  and  0  otherwise. 

Given  an  infinite  string  S  we  can  associate  with  it  two  languages  (S)  and 

K-(S): 

Definition  22.  For  an  arbitrary  oracle  infinite  string  S  we  define 
X+(S')  =  {s  6  {0, 1}*  :  si  is  an  initial  segment  of  S} 

and 

K-.(S)  =  {s  E  {0, 1}*  :  sO  is  an  initial  segment  of  5}. 

Example!.  Let  Ki  =  {1”0”  :  n  G  IN}.  Then  S{Ki)  is  the  infinite  string  con¬ 
sisting  of  O’s  only. 

K+{S{Ki))  =  0  and  K^iS{Ki))  =  {0}+. 

The  same  is  true  if  we  replace  Ki  by  i^2  =  {0^1”  :  n  G  IN}. 

Example  2.  Let  Kz  =  {01}*0.  Then  S{K3)  is  the  infinite  string  1000000000 . . .. 
K.^.(S{Ks))  =  {c},  i.e.  it  consists  of  the  empty  word  alone,  and  K-{S{Ks))  = 
{1(0*)}. 

More  generally,  we  have  the  following 

Observation 23.  Lei  jFC+  and  K-  he  two  disjoint  languages  in  {0,1}*  and 
Kiyi  =1,2  be  two  languages  with  K.^  Q  Ki  K-.  Then 

(i)  S{Ki)  =  S{K2)  and 

(ii)  for  S  =  S{Ki)  =  S{K2),  K^{S)  =  and  K-{S)  =  K.. 

Next  we  define,  given  an  infinite  string  5,  languages  L{S),  Li(S)  and  L2{S) 
as  follows: 

Notation  24.  For  any  number  x  we  denote  by  loglog{x)  the  function  log{log{x)) 
and  by  logloglog{x)  the  function  log{log{log{x))) 

Definition 25.  Let  a?  be  a  binary  word.  Let  bi(x)  =  and  62(2:)  = 

[log{l{x))loglog{l{x))\^  where  l(x)  denotes  the  length  of  x. 

For  an  arbitrary  infinite  string  5,  the  languages  L{S)  (Li(5),  L2{S))  are  defined 
as  follows:  x  G  L{S)  iff  the  /(aj)’th  (6i(a;)’th  62(x)^th)  bit  of  the  sequence  S'  is  1. 
If  S  =  S{K)  we  write  Lo{K)  instead  of  L(S{K))  and  similarly  for  Li  and  L2. 

We  note  the  following 

Observation  26. 

(i)  Lei  Si  and  S2  be  two  infinite  strings.  L{Si)  =  L{S2)  iff  Si  —  S2. 

(ii)  IfS  is  an  infinite  siring  and  x,  y  are  two  words  and  l{x)  =  l(y)  then  x  G  L{S) 
iffyeL{S). 

The  same  is  true  for  Li  and  L2. 

(Hi)  If  K  is  recognizable  by  a  polynomial  TM  then  so  is  Lo{K),  but  the  machine 
depends  on  K. 
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6  Capturing  Lq{K)  Uniformly 

We  now  discuss  the  complexity  of  Lq{K).  We  do  this  in  stages,  to  make  the  idea 
more  transparent. 

Lemma  27.  There  is  a  OTM  M  which  runs  in  polynomial  time,  such  that  Af(iir) 
recognizes  Lq{K)  uniformly. 

Proof.  M  keeps  one  tape  T  reserved  for  the  bits  of  S{K).  In  the  first  step  it 
writes  0  on  the  oracle  tape.  It  then  iterates  the  following  procedure:  query  the 
oracle,  if  it  accepts  add  a  1  to  T  else  add  a  0,  then  copy  the  contents  of  T  to  the 
oracle  tape.  In  each  iteration  the  head  over  the  input  tape  is  moved  one  position 
to  the  left,  when  all  the  input  is  scanned  accept  if  the  leftmost  bit  of  T  is  a  1. 

We  can  improve  upon  this  lemma,  using  relativized  uniform  circuits,  as  in¬ 
troduced  in  [Wil86]. 

Proposition  28.  There  is  an  Jj -uniform  family  of  oracle  boolean  circuits  (OBC) 
C  in  relativized  NC3  such  that  when  the  oracle  gates  of  C  recognize  strings  in 
K  Then  C  recognizes  Li{K). 

Proof  For  inputs  of  size  n  let  m  =  [log^{n)\ .  The  circuit  c„  is  built  from  exactly 
m  oracle  gates  connected  as  follows:  gate  1  has  a  single  input  0.  Every  gate  i>  1 
has  2—1  inputs  which  are  the  outputs  of  all  the  previous  gates.  The  output  of 
the  m’th  gate  is  the  output  of  the  circuit. 

Clearly,  given  m,  this  circuit  can  be  constructed  in  L.  Furthermore,  m  is 
L— computable  from  the  input,  so  this  family  of  circuits  is  L— uniform. 

Also  this  circuite  has  a  sublinear  number  of  gates,  and  is  of  a  depth  0{log'^(n)) 
oracle  gates.  As  each  oracle  gate  is  of  depth  log{k)  where  k  is  the  size  of  the 
input  to  that  gate,  this  gate  is  not  in  relativized  NC2,  but  is  in  relativized  NC3 
(it  depth  is  less  than  0{log^{n))). 

Also,  given  that  the  oracle  gates  recognize  a  set  K,  this  family  of  circuits 
recognizes  Li{K)  (the  output  for  each  gate  i  is  the  i’th  bit  of  S{K)). 

Using  the  same  idea,  but  working  harder,  we  get 

Theorem  29.  There  is  an  L-uniform  family  of  boolean  circuits  C  in  relativized 
NC2  such  that  when  the  oracle  gates  ofC  recognize  strings  in  K  Then  C  recog¬ 
nizes  L2{K). 

Proof  For  inputs  of  size  n  let  m  =  [log{n)loglog{n)\.  The  circuit  c„  is  built 
from  exactly  m  oracle  gates  connected  as  follows:  gate  1  has  a  single  input  0. 
Every  gate  2  >  1  has  2  —  1  inputs  which  are  the  outputs  of  all  the  previous  gates. 
The  output  of  the  m’th  gate  is  the  output  of  the  circuit. 

Given  that  the  oracle  gates  recognize  a  set  K,  this  family  of  circuits  clearly 
recognizes  L2{K)  (the  output  for  each  gate  2  is  the  2’th  bit  of  S{K)). 

Clearly  given  m  this  circuite  can  be  constructed  in  L  but  m  is  L-computable 
from  the  input,  so  this  family  of  circuits  is  L-uniform.  Also  this  circuite  has  a 
sublinear  number  of  gates. 


475 


Claim  30.  The  depth  of  these  circuits  is  0{log^(n)), 

Clearly  the  depth  of  the  circuit  is  exactly  m  oracle  gates.  As  each  oracle  gate  is 
of  depth  log{k)  where  k  is  the  size  of  the  input  to  that  gate,  the  depth  of  the 
circuite  is  given  by 

m  m 

t=2  ,*=1 

(The  first  gate  has  one  input  and  all  other  i  —  1  inputs) .  However 

m 

^log{i)  =  log{m\)  log(V2wmmT^ e~'^) 

1=1 

Where  the  approximation  is  via  Stirling’s  formula. 

log(V2^m'^e-^)  =  0{log{m))  +  mhg(m)  -  0{m)  =  0{mlog{m)) 
Substituting  for  m  we  get 

mlog{m)  —  {log(n)loglog{n))  •  log(log(n)loglog{n))  = 
{log{n)loglog(n))  •  {loglog{n)  -f  logloglog{n))  = 
0{log{n)loglog^{n))  <  0{log^(n)) 

7  The  Diagonalization 

We  are  now  ready  for  our  main  result. 

Theorem  31. 

(i)  For  no  set  of  oracles  does  £  =  FOL[Ki, . . . ,  K„,K]  capture 

uniformly. 

(ii)  In  particular,  cannot  be  uniformly  captured  by  formulas  of 

FOL[ATC,K]. 

Proof  For  an  arbitrary  oracle  K,  let  Lo{K)  be  as  in  definition  25  and  let  M{K) 
be  the  OTM  from  lemma  27  which  recognizes  Lq(^K)  uniformly.  To  prove  the 
theorem  it  is  sufficient  to  show  that  no  <I>{K)  e  S  defines  Lq{K)  uniformly.  Or 
in  other  words: 

Given  there  is  a  structure  A  and  two  sets  of  structures  K  and  K'  such 

that  A  e  L{M ,  K)  and  A  i  L{M,  K')  but  A  €  Mod{<j),  K)  and  A  G  Mod{()>,  K'). 

Assume,  for  contradiction,  that  (p{K)  G  ^{t)  is  r— sentence  which  defines 
Lo{K)  uniformly.  Let  A  be  a  r-structure  of  size  n  >  ,  where  is  from 

definition  20. 

We  are  going  to  construct  two  oracles  Kq,Ki,  depending  on  A,  such  that 
A  ^  Lo{Ko)  A  G  Lo{Ki).  The  oracles  will  differ  exactly  on  the  string  w  which 
is  used  by  M(K)  in  an  oracle  consultation,  i.e.  on  which  the  language  Lo{K) 
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depends,  but  which,  on  the  other  hand,  is  not  used  in  the  evaluation  oi  A  \=^ 

my 

In  the  definition  of  Lo(^)  we  use  the  infinite  string  5(iC),  cf.  definition  21. 
Let  S{K)j  denote  the  first  j  bits  of  S{K).  The  string  w  on  which  Kq  and  Ki 
will  differ  is  going  to  be  S{K)n-i> 

To  make  all  this  precise  we  prove  two  lemmas. 

The  first  lemma  is  obvious,  once  the  right  definitions  are  given. 

Lemma 32  (Dependency  Lemma).  Lei  Ki  be  an  oracle,  w  €  Ki  and  = 
K\  -  {u;}.  Lei  <j>£S  and  A  he  a  siruciure.  Ifw^ 

A  e  Mod{<j),Ki)  iffAe  Mod{cl>,K2)^ 

The  second  lemma  is  the  core  of  the  diagonalization;  It  is  proved  by  induction 
on  the  quantifier  depth. 

Lemma  33.  For  every  (p  €  S  ihere  is  an  no  €  IN  such  ihai  for  every  A  of  size 
n  bigger  ihan  no  ihere  is  an  oracle  Ka  wiih  S{KA)n-i  i  L<f>{A). 

In  fad,  no  =  ihe  diagonalizaiion  parameier  from  definiiion  20,  suffices. 

Proof  Let  n  denote  the  size  of  A  and  be  from  lemma  19.  We  prove  the  lemma 

by  induction  on  i  the  index  of  the  and  use  the  fact  that  there  are  only  d 
=Q-rank  of  (j>  many  of  them.  For  each  we  modify  our  oracle  K  (and  thus 
change  x  log{n)  bits  of  S{K))  such  that  all  queries  in  are  either  shorter 
than  i  X  ktp  X  log{n)  or  at  least  one  of  their  first  {i  x  k^p  x  log(n))  +  1  bits  is 
different  from  the  bit  in  the  same  place  in 
We  now  proceed  by  induction: 

Basis:  queries  depend  only  on  A.  We  choose  the  first  k^p  x  log{n)  bits  of 

S{K)  as  follows:  If  the  majority  of  queries  in  start  with  a  bit  1  we  chose  the 
first  bit  of  S{K)  to  be  0  and  vice  versa.  Clearly,  after  this  choice,  already  half 
of  the  queires  in  Ifp^\  are  in  the  desired  form.  For  the  j’th  bit  we  consider  only 
those  queries  of  which  match  S{K)  on  the  first  j  —  I  bits.  Among  those  if 
the  majority  starts  with  bit  x  we  choose  bit  x  (x  XOR  ^  =  1). 

Clearly,  after  k^  x  log{n)  bits  are  chosen,  all  queries  of  are  either  shorter 
than  i  X  k^  X  log{n)  or  at  least  one  of  their  first  (i  x  k<p  x  log{n))  +  1  bits  is 
different  from  the  bit  in  the  same  place  in  5'(7C)n_i  (which  is  as  yet  determined 
only  up  to  its  first  k^p  x  log{n)  bits). 

Note  that,  in  this  procedure,  we  have  determined  the  answers  only  for  those 
queries  which  were  possible  prefixes  of  S{K)  as  constructed  so  far.  For  the  other 
queries  in  answers  arbitrarily.  We  must  fix  these  answers  too,  if 

the  queries  in  7^,2  are  to  be  uniquely  determined. 

Induction  step:  Assume  we  have  succeeded  in  modifying  the  queries  and 
answers  of  7^,1  -7^, and  hence,  we  have  modified  the  first  (i- 1)  xk<px  log{n) 
bits  of  S{K).  Then  we  can  modify  7^,,*  as  follows: 

First  we  look  at  all  queries  of  7^,i  which  already  appeared  at  previous  levels. 
To  these  we  set  the  answers  they  had  in  the  previous  levels  (note  that  they 
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are  already  either  shorter  than  [i  —l)xktf,x  log(n)  or  at  least  one  of  their  first 
((i  —  1)  X  k(i,log{n))-\-l  bits  is  different  from  the  bit  in  the  same  place  in  S{K)n-i 
as  they  match  a  query  from  a  previous  level). 

For  new  queries  which  are  already  in  the  desired  form  we  set  arbitrary  an¬ 
swers. 

For  the  remaining  queries  We  ensure  our  condition  as  we  did  for  by 
iterating  the  following  procedure:  In  each  iteration  we  consider  only  queries  such 
that  their  first  ((i  —  1)  xk^px  log{Ti))  +  j  bits  coincide  with  the  determined  prefix 
of  S{K).  If  the  majority  of  these  have  x  as  their  ((i  —  1)  x  x  log{n))-\-j  -|-  I’th 
bit  we  choose  x  as  the  next  bit  of  5(i^),  as  in  the  basis  of  the  induction. 

Clearly,  after  k^  x  log{n)  such  iterations  all  queries  of  are  in  the  desired 
form.  As  before  for  those  queries  whose  answers  are  not  determined  during  this 
process  we  set  arbitrary  answers. 

After  we  have  completed  the  above  procedure  for  all  up  to  we 

have  determined  the  answers  of  an  oracle  Ka  to  all  queries  asked  for  checking 
A\=^  <j)  and  hence  we  have  determined  the  answer  of  the  model  checker  for  A  \=  (f) 
for  oracle  Ka-  However,  in  fact  we  have  determined  the  answer  of  A  \=  </>  for  any 
oracle  K  which  gives  the  same  answers  as  Ka  to  the  queries  in  =  Ui<*<d^ 
However,  none  of  this  polynomial  set  of  queries  is  S{K)n-i-  Hence,  using 
lemma  32,  we  have  the  freedom  to  select  K  and  K'  such  that  they  coincide  on 
all  asked  queries,  but  differ  on  S{K)n-i.  Therefore  either  ^  |=  or  ^  for 
both  K  and  K' .  But  A  E  Lq{K)  iff  ^  ^  Lo(K')  and  thus  we  prove  <j)  does  not 
uniformly  capture  Mk  ♦ 

Using  the  languages  Li{K)  or  L2{K)  from  proposition  28  and  theorem  29 
respectively,  we  can  actually  prove: 

Theorem  34. 

(i)  Let  Kiy ..  .^Km  he  a  fixed  set  of  oracles.  Let  8  =  FOL[Kif . . . ,  Km,  K]  be 
a  finitely  generated  Lindstrom  Logic  Li{K)  (L2{K))  be  the  oracle  languages 
from  section  5.  Then  no  <j>  E  8{Tdiag)  defines  Li(K)  (L2{K))  uniformly. 

(ii)  In  particular,  there  are  oracle  languages  Lo{K)  in  NCa^  (NC2^ )  which 
are  not  uniformly  definable  in  any  finitely  generated  Lindstrom  Logic. 

8  Conclusions  and  Conjectures 

We  have  introduced  the  notion  of  Logics  capturing  relativized  (=  oracle)  com¬ 
plexity  classes  uniformly  via  generalized  quantifiers.  We  have  shown  that  First 
Order  Logic  with  generalized  quantifiers  can  only  capture  comlexity  classes  re¬ 
lated  to  Logspace,  and  even  then  restrictions  on  the  oracle  computation  model 
have  to  made. 

On  the  other  hand,  Fixed  Point  Logics  and  Second  Order  Logic  are  capable 
of  capturing  quite  a  wide  class  of  relativized  polynomial  time  complexity  classes 
uniformly.  Our  main  result  states,  that  First  Order  Logic  cannot  do  this  for 
classes  containing  relativized  NC2. 
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It  seems  natural  to  ask,  whether  First  Order  Logic  is  not  rnore  appropriate 
for  capturing  relativized  classes  contained  in  NC2. 

We  end  this  paper  with  a  problem. 

Problem  35.  In  what  sense  does  FOL[K]  capture  relativized  ACq^  ? 

We  would  expect  a  positive  answer,  but  possibly  with  a  proviso.  It  would  fit 
Immerman’s  results  nicely,  cf.  [Imm89],  and  complement  our  results  in  this  paper. 
However,  these  low  complexity  classes  are  very  sensitive  to  minor  variations  in 
the  computational  model,  so  we  are  prepared  for  surprises. 
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Abstract.  We  develop  various  aspects  of  the  finite  model  theory  of 
L  (3)  and  X^^(3).  We  establish  the  optimality  of  normal  forms  for 
Loou;{'B)  over  the  cleiss  of  finite  structures  and  demonstrate  separations 
among  descriptive  complexity  classes  within  We  establish  neg¬ 

ative  results  concerning  preservation  theorems  for  X*(3)  and  XSoa^(3). 
We  introduce  a  generalized  notion  of  preservation  theorem  and  establish 
some  positive  results  concerning  “generalized  preservation  theorems”  for 
first-order  definable  classes  of  finite  structures  which  are  closed  under 
extensions. 


1  Introduction 

In  this  paper  we  investigate  the  status  of  preservation  theorems  in  finite  model 
theory.  We  focus  our  attention  on  classes  of  finite  structures  which  are  closed 
under  extensions  and  their  definability  in  fragments  of  the  infinitary  language 
The  language  T^^^was  introduced  by  Barwise  [4]  in  connection  with  the 
investigation  of  inductive  definability  over  infinite  structures.  Recently,  the  study 
-^coo^  played  a  central  role  in  analyzing  the  behavior  of  fixed-point  logics 
over  the  class  of  finite  structures  (see  [5,  13]).  Of  particular  interest  from  the 
point  of  view  of  our  current  investigation  are  the  works  of  Kolaitis  and  Vardi  [12] 
and  Afrati,  Cosmadakis,  and  Yannakakis  [1]  which  exploit  existential  fragments 
of  in  analyzing  the  expressive  power  of  Datalog. 

The  starting  point  for  our  investigation  is  the  well-known  failure  of  the  preser¬ 
vation  theorem  of  Los  and  Tarski  over  finite  structures.  Recall  that  the  Los- 
Tarski  Theorem  states  that  any  first-order  definable  class  of  structures  which  is 
closed  under  extensions  is  definable  by  a  first-order  existential  sentence.  Scott 
and  Suppes  conjectured  that  this  theorem  generalizes  to  the  finite  case,  that 
is,  if  Mod/(^)  (the  collection  of  finite  models  of  the  first-order  sentence  (p)  is 
closed  under  extensions,  then  Mod^(^)  =  Mody(^),  for  some  first-order  existen¬ 
tial  sentence  Tait  [18]  showed  that  this  conjecture  fails;  Gurevich  and  Shelah 
[9,  10]  gave  simpler  counterexamples  employing  universal-existential  first-order 
sentences. 

*  We  would  like  to  thank  Maria  Bonet,  Yuri  Gurevich,  and  Steven  Lindell  for  valuable 
discussions  on  the  subject  of  this  paper. 

**  Supported  in  part  by  NSF-STC  SBR-8920230. 
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In  light  of  the  failure  of  the  Los-Tarski  Theorem  over  finite  structures,  it  is 
natural  to  inquire  whether  “generalized  preservation  theorems”  might  hold  in  the 
finite  case.  In  this  paper,  we  investigate  the  prospects  for  such  a  positive  approach 
to  preservation  properties  in  the  context  of  finite  model  theory.  In  particular, 
we  examine  generalized  versions  of  ordinary  preservation  theorems  where  an 
algebraic  restriction  on  a  class  of  structures  definable  in  a  given  language  yields 
information  about  the  syntactic  structure  of  formulas  which  define  that  class  in 
an  extension  of  that  language.  In  this  spirit,  we  show  that  for  certain  classes  of 
first-order  sentences  <?,  if  9?  G  ^  and  Mody(^)  is  closed  under  extensions,  then 
Mod/(^)  =  Mod/(^)  for  some  ^  in  the  existential  fragment  of  (or  even 
in  Datalog(7^,  -•)).  In  contrast,  we  also  establish  the  failure  of  the  analog  of  the 
Los-Tarski  Theorem  for  itself,  both  over  finite  structures  and  over  arbitrary 
structures.  That  is,  we  show  that  there  is  a  sentence  ip  of  such  that  both 
Mod/(^)  and  Mod(<^)  are  closed  under  extensions,  but  neither  of  these  classes 
is  definable  by  an  existential  sentence  of 

The  paper  proceeds  as  follows.  The  next  section  introduces  the  languages 
we  will  study  and  establishes  a  simple  proposition  which  characterizes  the  rel¬ 
ative  expressive  power  of  their  existential  fragments.  Section  3  develops  some 
finite  model  theory  for  the  existential  fragments  of  and  In  particu¬ 

lar,  we  establish  the  optimality  of  a  normal  form  for  the  existential  fragment 
of  over  finite  structures  and  demonstrate  separations  among  descriptive 
complexity  classes  within  In  Section  4,  we  prove  the  failure  of  existen¬ 
tial  preservation  for  Section  5  is  devoted  to  establishing  positive  results 

concerning  generalized  preservation  theorems  for  fragments  of  first-order  logic 
over  finite  structures.  In  the  final  section,  we  discuss  a  number  of  open  problems 
and  present  without  proof  some  related  results  concerning  preservation  under 
homomorphisms.  A  full  treatment  of  these  results  will  appear  in  [17]. 

2  Preliminaries 

Let  be  the  collection  of  finite  structures  of  signature  a.  We  will  assume 
that  the  universe  of  any  A  G  is  an  initial  segment  of  iV  =  {0,1,2,...}.  We 
will  often  use  A,  B, ...  etc.  to  denote  both  a  structure  and  its  universe  when  no 
confusion  is  likely  to  result.  We  assume  that  the  signature  cr  is  finite  and  contains 
no  function  symbols;  we  suppress  mention  of  cr  when  no  confusion  is  likely  to 
result.  A  boolean  query  C  C  is  a  class  of  finite  structures  that  is  closed  under 
isomorphisms.  We  use  C  to  range  over  boolean  queries.  In  what  follows,  we  will 
focus  attention  on  boolean  queries  which  are  closed  under  extensions. 

Definition!.  EXT  =  {C  C  JF  |  VA,  B  G  C,  if  A  G  C  and  A  C  B,  then  B  G  C}. 

Let  L  be  a  logical  language  and  let  y?  be  a  sentence  of  L.  Mod(^)  =  {A  | 
A  1=  is  the  L-class  determined  by  9?  and  Mod/(^)  =  {A  G  ^  |  A  [=  9?}  is  the 
boolean  query  expressed  by  9?.  We  say  that  C  is  L-definable^  just  in  case  it  is  the 
boolean  query  expressed  by  some  sentence  ip  €  L.  We  will  often  use  L  to  denote 
the  set  of  L-definable  boolean  queries.  We  let  FO  denote  first-order  logic,  L^owi 
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the  usual  infinitary  extension  of  first-order  logic  which  allows  conjunction  and 
disjunction  over  arbitrary  sets  of  formulas,  ,  the  fragment  of  FO  consisting  of 
those  formulas  all  of  whose  variables  both  free  and  bound  are  among  xi, . . . , 
and  similarly  the  fc-variable  fragment  of  loou.;  We 

let  FO(3)  denote  the  set  of  existential  formulas  of  FO,  that  is,  those  formulas 
obtained  by  closing  the  set  of  atomic  formulas  and  negated  atomic  formulas 
under  the  operations  of  conjunction,  disjunction,  and  existential  quantification. 
We  define  Loow(3),  the  set  of  existential  formulas  of  Loow,  similarly,  but  require, 
in  addition,  closure  under  infinitary  conjunction  and  disjunction.  We  let  L^(3) 
consist  of  the  formulas  common  to  FO(3)  and  L*  and  we  define  and 

^oow(3)  similarly.  A  Datalog(/,  -i)  program  P  is  a  collection  of  rules  of  the  form 

T]0  < — 

Such  a  rule  has  a  head,  tjq,  and  a  body,  t}i,  . . ,  Each  of  the  rji  is  either  an  in¬ 
equality  or  a  literal  over  the  signature  crUr  where  <t  and  r  are  disjoint;  a  consists 
of  the  exiensional  relations  and  constants  of  P  and  r  consists  of  the  iniensional 
relations  of  P.  The  heads  of  all  rules  are  built  from  intensional  relations  and  in- 
tensional  relations  occur  only  positively  throughout  P.  The  program  contains  a 
distinguished  intensional  relation  R  of  arity  n  >  0  and  determines  an  n-ary  query 
over  structures  in  Ta  •  The  value  of  this  query  for  a  given  A  To  is  the  value  of 
R  when  the  program  is  viewed  as  determining  least-fixed  points  for  each  of  the 
intensional  relations  with  respect  to  a  simultaneous  induction  associated  with 
the  program.  The  reader  may  consult  [1,  12]  for  further  details  and  discussion. 
As  with  logics,  we  use  Datalog(^,  -n)  to  refer  to  the  class  of  queries  computed  by 
Datalog(^,  -i)  programs  as  well  as  to  the  class  of  programs  themselves.  Datalog 
programs  are  defined  similarly  except  that  all  the  r}i  are  restricted  to  be  positive 
literals,  even  those  built  from  extensional  relations.  Observe  that  Datalog(7^,  -») 
is  contained  in  the  least  fixed-point  extension  of  first-order  logic  (FO+LFP). 

In  our  current  notation,  the  failure  of  the  Los-Tarski  Theorem  over  finite 
structures  may  be  expressed  as: 

FO  n  EXT  g  FO(3). 

This  raises  the  question  of  whether  FO  n  EXT  is  contained  in  the  existential 
fragment  of  some  stronger  logic.  The  following  proposition  completely  charac¬ 
terizes  the  relative  expressive  power  of  the  existential  fragments  of  the  logics  in 
which  we  are  interested. 

Proposition  2. 

F0(3)  C  Data]og(/,  -)  C  i"  „(3)  C  iooa,(3)  =  EXT. 

Proof.  It  is  easy  to  see  that  every  query  in  FO(3)  can  be  expressed  by  a  program 
in  Datalog(^,  ->)  which  makes  use  of  no  recursion.  It  is  well-known  that  this 
inclusion  is  strict,  for  example,  the  query  (s,  f )-connectivity  is  expressible  in 
Datalog  but  not  in  FO.  The  inclusion  of  Datalog(/,  -i)  in  ^^<^(3)  has  been  noted 
by  Afrati,  Cosmadakis,  and  Yannakakis.[l]  (see  also  [12]);  the  argument  to  show 
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this  is  a  variant  of  the  proof  that  least  ftxed-point  logic  is  contained  in  over 
the  class  of  finite  structures  (see  [14]).  Afrati,  Cosmadakis,  and  Yannakakis  [1] 
also  exhibit  queries  which  witness  the  separation  of  Datalog(^, -i)  and 
even  over  the  class  of  polynomial  time  computable  queries.  The  identity  between 
f>oow(3)  and  EXT  has  been  noted  by  Kolaitis  and  independently  by  Lo  (see  [1] 
and  [15]).  Finally,  it  is  easy  to  construct  polynomial  time  computable  boolean 
queries  in  EXT  which  are  not  in  For  example,  let  C  be  the  query  over  the 
signature  {E,  s,  t}  of  source-target  graphs  that  says  that  there  is  an  E’-path  from 
s  to  t  whose  length  is  less  than  half  the  cardinality  of  the  structure.  It  is  clear 
that  C  G  EXT.  It  is  also  easy  to  verify  that  C  is  not  in  (and  therefore  not 
in  L^(^(3))  by  a  straightforward  application  of  the  fc-pebble  Ehrenfeucht-Fraisse 
game  which  we  review  below.  ■ 

The  above  proposition  together  with  the  failure  of  the  Los-Tarski  Theorem 
in  the  finite  case  suggests  the  following  questions. 

1.  Is  FO  n  EXT  C 

2.  Is  FO  n  EXT  C  Datalog(9i^,  -i)? 

3.  Is  L-a,nEXTCL^,(3)? 

Clearly  a  positive  answer  to  the  second  or  third  question  would  imply  a  positive 
answer  to  the  first.  In  Section  4,  we  provide  a  negative  answer  to  the  third 
question.  In  Section  5,  we  provide  partial  positive  answers  to  the  first  and  second 
questions.  Before  proceeding  to  these  results,  we  develop  some  of  the  finite  model 
theory  of  L^{3)  and  jb^a;(3)  in  the  next  section. 

3  Basic  Finite  Model  Theory  for  ^*^(3)  and  i^y(3) 

In  this  section,  we  present  some  basic  model  theory  for  i*,  -^*(3)1  “<1 

LJq^^,(3).  After  a  brief  discussion  of  game-theoretic  characterizations  of  equiv¬ 
alence  and  definability  in  these  languages,  we  proceed  to  consider  questions  of 
finite  axiomatizability  and  normal  forms. 

Let  L  be  one  of  the  logical  languages  we  consider.  Given  a  structure  A,  the  L- 
iheory  of  A  is  the  collection  of  sentences  of  L  which  are  satisfied  by  A.  We  say  that 
A  is  L- equivalent  to  J5,  if  and  only  if,  the  L-theory  of  A  is  equal  to  the  L- theory 
of  B  and  we  say  that  A  is  L- compatible  with  B,  if  and  only  if,  the  L-theory  of  A  is 
contained  in  the  L-theory  of  B.  Note  that  if  L  is  closed  under  negation,  then  the 
relations  of  L-equi valence  and  L-compatibility  coincide,  whereas  for  languages 
like  L^{3)  and  L^j^(3)  these  relations  are  distinct.  We  use  the  notations  =*, 
and  for  L* -equivalence,  Lj^^^-equivalence,  L*(3)-compatibility, 
and  L^j^(3)-compatibility,  respectively.  The  main  tool  for  studying  these  rela¬ 
tions  are  refinements  of  the  Ehrenfeucht-Fraisse  game.  Barwise  [4]  characterized 
-equivalence  in  terms  of  partial  isomorphisms,  while  Immerman  [11]  and 
Poizat  [16]  provided  related  pebble  game  characterizations  of  L*' -equivalence. 
Kolaitis  and  Vardi  [12]  characterized  compatibility  in  the  negation  free  fragment 
of  L^4^(3)  both  in  terms  of  collections  of  partial  homomorphisms  as  well  as  in 
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terms  of  a  one-sided,  positive  version  of  the  pebble  game.  Below  we  use  a  minor 
variant  of  the  approach  in  [12]  to  characterize  X (3)-compatibility. 

A  collection  I  of  partial  isomorphisms  from  A  to  B  is  said  to  have  the  k~ 
\hach-and-^foTih  property  if  for  all  /  G  /  such  that  the  domain  of  /  has  cardinality 
<  and  all  a  G  A  [6  G  5],  there  is  a  function  g  E  I  such  that  f  Q  g  and  a  G 
dom(ijf)[6  G  rng(5f)].  (That  is,  the  A?-forth  property  is  the  one-sided  version,  going 
forth  from  A,  of  the  ^-back-and-forth  property.) 

Barwise  [4]  proved  the  following  proposition  which  gives  an  algebraic  char¬ 
acterization  of  L^j^-equivalence. 

Propositions  (Barwise  [4]).  Lei  A  and  B  be  structures  of  signature  cr  and 
let  h  be  the  map  with  dom(/i)  =  {c^  |  c  G  cr}  such  that  h{c^)  =  c^  for  all  c  E  a. 
The  following  conditions  are  equivalent. 

1-  a=1^b. 

2.  There  is  a  non-empty  set  I  of  partial  isomorphisms  from  A  to  B  such  that 

(a)  I  is  closed  under  subfunctions; 

(b)  I  has  the  k -back- and- forth  property; 

( c)  for  all  f  E  I,  f  U  h  is  a  partial  isomorphism  from  A  to  B. 

In  a  similar  spirit,  Kolaitis  and  Vardi  [12]  gave  an  algebraic  characterization 
of  the  compatibility  relation  for  the  negation  free  fragment  of  T^^(3)  in  terms 
of  collections  of  partial  homomorphisms  with  the  ib-forth  property.  We  adapt 
their  approach  to  the  case  of  in  the  following  theorem. 

Proposition4  (Kolaitis  and  Vardi  [12]).  Let  A  and  B  be  structures  of  sig¬ 
nature  <T  and  let  h  be  the  map  with  dom{h)  =  {c^  |  c  G  o'}  such  that  h(c^)  =  c^ 
for  all  c  E  (T.  The  following  conditions  are  equivalent. 

2.  There  is  a  non-empty  set  I  of  partial  isomorphisms  from  A  to  B  such  that 

(a)  I  is  closed  under  subfunctions; 

(b)  I  has  the  k-forth  property; 

( c)  for  all  f  E  I,  f  U.h  is  a  partial  isomorphism  from  A  to  B. 

Both  Propositions  3  and  4  can  be  expressed  more  colorfully  in  terms  of  pebble 
games.  This  approach  to  L* -equivalence  was  introduced  by  Immerman  [11]  and 
Poizat  [16]  and  as  an  approach  to  TjQ^^(3)-compatibility  by  Kolaitis  and  Vardi 
[12].  In  order  to  state  the  relevant  results  in  a  suitably  refined  form,  we  require 
the  notion  of  the  quantifier  rank  of  a  formula.  We  state  this  definition  for  formulas 
of  Tooa;  since  all  the  languages  we  consider  are  fragments  of  it. 

Definitions.  The  quantifier  rank  of  ^  G  Loou>  (qr(^))  is  defined  by  the  following 
induction. 

1.  qr(^)  =  0  if  V?  is  atomic; 

2.  qr(-i9?)  =  qi{(py, 

3.  qr(A^)  =  qr(V ^)  =  sup({qr(v?)  |  <p  E  ^}); 
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4.  qr(3ic^)  =  qr(Va?^)  =  qr(y?)  +1. 

The  n-round,  k-pebble  Ehrenfeuchi-Fraisse  game  on  A  and  B  is  played  be¬ 
tween  two  players,  Spoiler  and  Duplicator,  with  k  pairs  of  pebbles,  (ai,  /?i), . . . , 
(ttfcjft)-  The  Spoiler  begins  each  round  by  choosing  a  pair  of  pebbles  (a*,/?*) 
that  may  or  may  not  be  in  play  on  the  boards  A  and  B.  He  (by  convention,  the 
Spoiler  is  male,  the  Duplicator  female)  either  places  a,-  on  an  element  of  A,  or  ft 
on  an  element  of  B.  The  Duplicator  then  plays  the  remaining  pebble  on  the  other 
model.  The  Spoiler  wins  the  game  if  after  any  round  m  <n  the  function  /  from 
A  to  B,  which  sends  the  element  pebbled  by  a,-  to  the  element  pebbled  by  ft  and 
preserves  the  denotations  of  constants,  is  not  a  partial  isomorphism;  otherwise, 
the  Duplicator  wins  the  game.  The  n-round  3^-game  is  the  one-sided  version 
of  the  n-round,  A;-pebble  Ehrenfeucht-Fraisse  game  in  which  the  Spoiler  is  re¬ 
stricted  to  play  a  pebble  ai  into  A  at  every  round  while  the  Duplicator  responds 
by  playing  ft-  into  H;  the  winning  condition  remains  the  same.  Both  the  A;-pebble 
Ehrenfeucht-Fraisse  game  and  its  one-sided  variant  have  infinite  versions,  which 
we  call  the  eternal  fc-pebble  Ehrenfeucht-Fraisse  game  and  the  eternal  3^-game. 
In  these  games,  the  play  continues  through  a  sequence  of  rounds  of  order  type 
u.  The  Spoiler  wins  the  game,  if  and  only  if,  he  wins  at  the  n^^-round  for  some 
n  E  w  as  above;  otherwise,  the  Duplicator  wins.  In  describing  the  play  of  pebble 
games  below,  we  will  often  use  S  to  refer  to  the  Spoiler  and  D  to  refer  to  the 
Duplicator.  We  will  also  often  use  Q:i,ft,  etc.  to  refer  to  both  pebbles  and  the 
elements  they  pebble  at  a  given  round  of  play. 

The  foregoing  n-round  games  may  be  used  to  characterize  equivalence  and 
compatibility  of  structures  with  respect  to  sentences  and  L*(3)  sentences 
of  quantifier  rank  n,  and  the  eternal  games  may  be  used  to  characterize  equiva¬ 
lence  and  compatibility  of  structures  with  respect  to  sentences  and  L^^(3) 
sentences.  Given  structures  A  and  B  we  let  A=^’^B,  if  and  only  if,  A  and  B 
satisfy  the  same  sentences  of  of  quantifier  rank  <  n  and  we  let  A^^^^B^  if 
and  only  if,  every  sentence  of  L^(3)  of  quantifier  rank  <  n,  which  is  true  in  A,  is 
also  true  in  B.  The  following  two  propositions  use  the  n-round  pebble  games  to 
characterize  these  relations.  The  first  is  due  to  Immerman  [11]  and  Poizat  [16] 
and  the  second  is  essentially  due  to  Kolaitis  and  Vardi  [12]. 

PropositionG  (Immerman  [11],  Poizat  [16]).  For  all  structures  A  and  B, 
the  following  conditions  are  equivalent, 

1.  A=^^^B.  ■ 

2.  The  Duplicator  has  a  winning  strategy  for  the  n-round^  k-pebble  Ehrenfeucht- 
Fraisse  game  on  A  and  B. 

Proposition?  (Kolaitis  and  Vardi  [12]).  For  all  structures  A  and  B,  the 
following  conditions  are  equivalent. 

1. 

2.  The  Duplicator  has  a  winning  strategy  for  the  n-round  3^ -game  on  A  and 
B,  with  the  Duplicator  playing  on  B. 
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The  next  proposition  gives  a  characterization  of  the  infinitary  equivalence 
and  compatibility  relations  in  terms  of  the  eternal  games.  It  is  essentially  due  to 
Kolaitis  and  Vardi  [14,  12]. 

Propositions  (Kolaitis  and  Vardi  [14,  12]).  L  For  all  structures  A  and 
Bj  the  following  conditions  are  equivalent. 

(a)  A=l^B. 

(h)  The  Duplicator  has  a  winning  strategy  for  the  eternal  k-pebble  Ehrenfeucht- 
Fraisse  game  on  A  and  B. 

2.  For  all  structures  A  and  5,  the  following  conditions  are  equivalent. 

(b)  The  Duplicator  has  a  winning  strategy  for  the  eternal  3^ -game  on  A  and 
B,  with  the  Duplicator  playing  on  B. 

Kolaitis  and  Vardi  [14,  12]  observed  that  over  finite  structures  infinitary 
equivalence  and  compatibility  coincide  with  their  finitary  analogs. 

Proposition9  (Kolaitis  and  Vardi  [14,  12]).  1.  Let  A  orB  be  a  finite  struc¬ 
ture.  Then,  the  following  conditions  are  equivalent. 

(a)  A=t,^B. 

(h)  A='‘B. 

2.  Let  B  be  a  finite  structure.  Then,  the  following  conditions  are  equivalent. 

(a)  A^l^B. 

(b)  A:<^B. 

The  foregoing  propositions  yield  the  following  corollaries  concerning  defin¬ 
ability. 

Proposition  10  (Kolaitis  and  Vardi  [12]).  For  all C  C  the  following  con¬ 

ditions  are  equivalent. 

1.  C  is  LIq^(3)- definable. 

2.  For  allAeC  and  B^C, A:^l^^B. 

3.  For  all  AeC  and  B  ^C,A:^^B. 

i.  For  all  A  €  C  and  B  ^  C,  there  is  an  n  ^  u  such  that  the  Spoiler  has 
a  winning  strategy  for  the  n-round  3^ -game  on  A  and  B  with  the  Spoiler 
playing  on  A. 

Let  L  and  U  be  logical  languages  and  let  T  be  a  collection  of  sentences  of  L. 
We  say  that  T  is  finitely  axiomaiizable  in  V ,  if  and  only  if,  there  is  a  sentence 
(p  E  U  such  that  Mod/(T)  =  Mod/(y?).  Dawar,  Lindell  and  Weinstein  [5]  prove 
that  the  Lj^^^-theory  of  any  finite  model  is  finitely  axiomatizable  in  L^.  As  a 
corollary,  they  obtain  a  simple  normal  form  for  over  F,  in  particular,  they 
show  that  every  sentence  of  L^^^  is  equivalent  to  a  countable  disjunction  of 
sentences  of  L*  and  is  also  equivalent  to  a  countable  conjunction  of  sentences  of 
L^.  In  contrast,  we  show  below  that  there  are  finite  models  whose  L*(3)-theories 
are  not  finitely  axiomatizable  in  L^(3).  Building  on  this  result,  we  prove  that  the 
normal  form  for  over  !F  (every  sentence  of  is  equivalent  over  to  a 
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countable  disjunction  of  countable  conjunctions  of  sentences  of  exhibited  by 
Kolaitis  and  Vardi  [14]  is  optimal  when  considered  as  a  normal  form  for 
sentences  over 

We  begin  by  proving  that  there  are  models  whose  L*(3)-theories  are  not 
finitely  axiomatizable  in  ^>*(3).  Our  argument  exploits  the  k-extension  axioms^ 
which  we  now  describe  briefly.  Let  cr  be  a  purely  relational,  finite  signature.  A 
basic  k-iype  tt  over  the  signature  o-  is  a  maximal  consistent  set  of  literals  over  a  in 
the  variables  xi, . . , ,  xjb.  A  k-exiension  axiom  of  signature  o-  is  a  sentence  of  the 
form  Vxi . .  .Xk~i3xk{f\7r  where  tt  is  a  basic  {k  —  l)-type  of  signature 

<7,  tt'  is  a  basic  k-type  of  signature  o',  and  tt  C  tt'.  Over  a  fixed  signature  (T,  the  k- 
Gaifman  theory^  Fk,  is  the  set  of  all  ^-extensions  axioms  of  signature  o*.  It  is  easy 
to  see  that,  for  each  fc,  there  are  only  finitely  many  A:-extension  axioms.  Gaifman 
[7]  showed  that  the  theory  T  =  (Jfc  A  axiomatizes  an  w- categorical  model  called 
the  random  structure,  Fagin  [6]  proved  the  0-1  law  for  first-order  logic  by  showing 
that  every  extension  axiom  is  almost  surely  true  over  F.  Fagin’s  result  implies 
that  almost  every  A  £  F  satisfies  the  Ar-Gaifman  theory.  Immerman  [11]  showed 
that  any  two  models  of  the  /:-Gaifman  theory  are  L*' -equivalent  and  Kolaitis  and 
Vardi  [14]  made  use  of  the  fc-Gaifman  theory  in  their  proof  of  the  0  —  1  law  for 
make  the  following  easy  observation. 

Proposition  11.  Let  A\=  Fk,  and  let  B  be  any  (finite  or  infinite)  model.  Then 
B:<^^^A,  Equivalently,  for  all  (p  6  L^^(3),  if  (p  is  satisfiable,  then  A\=  p. 

Proof.  The  proof  follows  easily  from  Proposition  8  by  considering  the  eternal 
3*^ -game  on  B  and  A  with  the  Duplicator  playing  on  A.  The  ^-Gaifman  axioms 
essentially  say  that  D  can  extend  a  partial  isomorphism  with  domain  of  size  <  k 
in  every  possible  way.  Therefore,  she  has  a  winning  strategy  for  the  game.  ■ 

We  observe  that  this  result  yields  a  compactness  theorem  over  finite  struc¬ 
tures  and  a  finitary  analog  of  the  Lowenheim-Skolem  Theorem  for  L^j^(3). 

Corollary  12.  For  every  k  there  is  an  Uk  such  that  for  every  set  0  of 
sentences  of  0  is  satisfiable,  if  and  only  if,  every  finite  subset  of  0  is 

satisfiable,  if  and  only  if,  0  is  satisfied  in  a  model  of  size  rik. 

The  next  proposition  establishes  that  there  are  finite  structures  whose  L*(3)- 
theory  is  not  finitely  axiomatizable  in  L*(3). 

Proposition  13.  For  all  k  >2,  there  is  a  model  Ak  £  F  such  that  the  L^(3)- 
theory  of  Ak  is  not  finitely  axiomatizable  in  L^(3). 

Proof.  Let  Ak  be  any  finite  model  of  the  Ar-Gaifman  theory  over  the  language 
of  graphs.  We  show  that  for  any  n  €  w,  there  is  a  such  that 
and  This  implies  that  the  theory  of  Ak  cannot  be  axiomatized 

by  L^{3)  sentences  of  quantifier  rank  <  n  and,  therefore,  that  it  is  not  finitely 
axiomatizable  in  L*  (3). 

For  the  purpose  of  defining  the  models  SJ,  we  require  the  following  notion 
and  notation.  A  basic  ^-type  tt  satisfies  the  distinctness  condition  if  for  every 
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/  <  Ar,  the  formula  x/  7^  6  Let  {tti,  . . . ,  tt,}  be  a  set  of  basic  {k  —  l)-types 

such  that 

1.  every  basic  {k  —  l)-type  is  equivalent  to  some  tt,-  and 

2.  if  z  7^  j,  then  tt,-  is  not  equivalent  to  tTj-. 

Similarly,  for  each  1  <  i  <  s,  let  {tt,-,!,  . . . ,  be  a  set  of  basic  fc-types  each 

of  which  extends  tt*  and  satisfies  the  distinctness  condition  such  that 

1.  every  basic  i-type  which  extends  tt,-  and  satisfies  the  distinctness  condition 
is  equivalent  to  some  TVij  and 

2.  if  j  7^  /,  then  iTij  is  not  equivalent  to  7r,j/. 

We  proceed  to  define  the  models  .  Let  be  the  graph  on  two  vertices 
with  exactly  one  loop  and  no  other  edges.  Thus  Bl  realizes  both  basic  1-types. 
Given  that  B^  has  been  defined,  we  now  define  B^^^  as  an  extension  of  B^. 
For  each  —  l)-tuple  b  of  elements  of  B^,  let  r(6)i  be  the  unique  i  such  that 
N  3.nd  let  Xj  =  I  1  ^  ^  21  set  of  distinct  objects 

disjoint  from  We  suppose  that  for  any  distinct  pair  of  {k  -  l)-tuples  a  and 
b  of  elements  of  B^yXaH  Xj-  =  0.  Let  X  be  the  union  of  all  the  sets  X-^.  We 
let  the  universe  of  B^'^^  =  B^  U  X.  The  edge  relation  of  B^'^^  is  obtained  from 
that  of  B^  by  adding  the  minimal  number  of  edges  so  that  each  ib-tuple  6  * 

satisfies  well-defined.  We  say  that  the 

height  of  an  element  6  introduced  in  this  construction  is  the  least  n  such  that 
beB^, 

We  first  show  that  B^ ,  By  Proposition  7,  it  suffices  to  describe  a 

winning  strategy  for  D  in  the  n-round  3*^-game  with  D  playing  on  B^  and  S 
playing  on  Ak^  The  strategy  we  describe  for  D  will  allow  her  to  play  her 
move  on  some  b  G  B^^  for  each  m  <  n.  In  round  1,  D  answers  the  first  move 
of  S  by  playing  her  pebble  on  the  appropriate  element  of  Bl  C  B^  to  create 
a  partial  isomorphism.  Suppose  that  D  has  played  only  onto  elements  of  B^ 
through  round  m,  where  m  <  n.  Let  S  choose  pebble  pair  (a/,/?/)  to  play  in 
round  (m-f  1).  We  consider  two  cases.  If  S  plays  ai  on  the  same  element  as  some 
ot\t ,  for  /  ^  /',  then  D  must  play  (3}  onto  the  element  pebbled  by  .  Doing  so,  she 
obviously  maintains  a  partial  isomorphism  and  succeeds  in  playing  within  . 
On  the  other  hand,  suppose  that  S  plays  ai  on  a  distinct  element  such"  that  the 
elements  pebbled  hya*aj  on  A  after  the  round  satisfy  7r,  j  (we  may  need  to  pad 
the  tuple  pebbled  by  a  to  a  tuple  of  length  (k  —  l)  by  repeating  its  last  element,  if 
all  the  pebbles  are  not  in  play  at  this  round).  Before  D  plays  her  (m-h  1)^^  move, 
the  pebbles  /?  are  on  a  tuple  b  (similarly  padded,  if  necessary)  that  satisfies  tt^. 
She  then  plays  /?/  on  the  element  G  B^'^^  ^  thereby  maintaining  a  partial 
isomorphism.  This  strategy  enables  her  to  win  the  n-round  game. 

Next,  we  show  that  .  By  Proposition  7,  it  suffices  to  show  that 

S  can  win  the  (n  -h  l)-round  game  with  D  playing  on  B^  and  S  playing  on  Ak. 
We  describe  a  strategy  for  play  by  S  which  forces  D  to  pebble  an  element  of 
height  at  least  m  by  the  end  of  round  m  to  avoid  losing  at  that  round.  It  follows 
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that  S  wins  the  (n  +  l)-round  game  since  all  elements  of  have  height  <  n, 
S  plays  as  follows.  He  first  places  his  ^-pebbles  on  a  set  of  k  distinct  elements 
which  form  a  Ar-clique,  that  is,  for  every  pair  of  distinct  pebbled  elements  a  and 
a',  Ak  1=  E{a,a').  S  may  play  in  this  way  since  Ak  [=  A-  By  our  construction 
above,  if  b,b'  ^  are  distinct  elements  of  the  same  height,  E(b,E).  It 

follows  immediately  that  any  r-clique  in  Bjf  contains  an  element  of  height  at 
least  r.  Therefore,  if  S  has  not  won  by  round  k^  D  has  pebbled  an  element  of 
height  at  least  k  by  the  end  of  that  round.  Note  that  in  case  (n  + 1)  <  fe,  we  are 
done,  since  at  round  (n  +  1),  D  will  be  unable  to  play  onto  an  element  of  height 
at  least  (n  +  1)  to  form  an  (n  +  l)-clique. 

We  proceed  to  describe  the  strategy  for  S’s  continuing  play  under  the  as¬ 
sumption  that  k  <  {n  A  1).  Suppose  that  through  round  m,  ^  <  m  <  (n  -f  1),  D 
has  played  a  pebble  onto  an  element  of  height  at  least  m,  and  that  the  k  pebbles 
S  has  played  lie  on  distinct  elements  of  ^4*  which  form  a  fc- clique.  We  show  how 
S  can  play  to  ensure  that  D  must  play  onto  an  element  of  height  at  least  (m  -|- 1) 
at  round  (m  -f  1),  if  she  is  to  prevent  S  from  winning  at  this  round,  and  leave 
the  round  with  a  fc-clique  pebbled.  Suppose  that  Pi  is  pebbling  an  element  b  of 
height  greater  than  the  height  of  any  other  element  pebbled  in  at  round  m. 
By  our  hypothesis,  the  height  of  6  is  at  least  m.  Pick  j  ^  i  (recall  that  2  <  k) 
and  let  a  €  Ak  he  the  element  pebbled  by  aj .  S  picks  up  cxj  and  places  it  on  an 
a'  E:  Ak  such  that 

1.  Ak  1=  E{a,  a)  ^  ■~>E{a\  a')  and 

2.  for  every  a"  G  Ak  on  which  one  of  the  remaining  (Ar  - 1)  pebbles  lies,  a'  ^  a" 

and  Ak  ^E{a\a")AE(a”,a'). 

The  existence  of  such  an  a'  follows  from  the  fact  that  Ak  |=  A-  We  claim  that 
to  avoid  losing  at  this  round,  D  must  play  her  pebble  Pj  onto  an  element  6'  of 
height  greater  than  the  height  of  6,  and  hence  of  height  at  least  (m  +  1).  Let  6" 
be  the  element  pebbled  by  Pj  at  round  m.  By  our  construction,  each  element  of 
B^  is  connected  to  at  most  {k  -  1)  elements  of  lesser  height.  Therefore,  from  the 
hypotheses  that  S  had  pebbled  a  Ar-clique  at  round  m,  and  that  b  is  an  element 
of  maximal  height  pebbled  by  D  at  that  round,  we  may  conclude  that  the  only 
element  of  height  <  the  height  of  b  adjacent  to  b  onto  which  D  could  play  Pj 
is  b”  itself.  But  this  play  would  fail  to  maintain  a  partial  isomorphism  with  the 
elements  S  has  now  pebbled  at  round  (m  -{-  1)  by  the  first  condition  we  have 
imposed  on  the  choice  of  a'  above.  Therefore,  to  avoid  losing  at  round  (m  -f  1), 
D  must  pebble  an  element  of  height  at  least  (m  H- 1).  ■ 

The  next  result  follows  immediately. 

Corollary  14,  There  are  infinitely  many  formulas  of  (3)  which  are  pairwise 
inequivaleni  over  T . 

We  now  consider  L*o^^(3)-theories  and  normal  forms  for  sentences 

over  T.  We  let  Thg(A)  denote  the  (3)- theory  of  A.  Before  proceeding,  we 
define  the  following  fragments  of  L^j^(3). 


490 


1.  Let  /\L^{3)  =  {0\9  =  for  some  ^  C  L^(3)}. 

2.  Let  V ^*(3)  =  {^  I  ^  =  V^»  for  some  ^  C  L*(3)}. 

3.  Let  ^(V  L^{3))  =  {0  I  0  =  for  some  countable  ^  C\/  L*(3)}. 

4.  Let  Y(/\L*(3))  =  {0  I  0  =  \/^,  for  some  countable  0  C  /\L*'(3)}. 

Proposition  15.  For  all  finite  structures  A,  there  is  a  0  £  /\L*(3)  such  that 
Mod;(^)  =Mod/(Th^(^)). 

Proof.  Observe  that  Mod/(Th3(i4))  =  {B  £  \  A:<!^^^B}.  Let  Ca  ~  ^  — 

Mod/(Thg(vl)).  By  Proposition  9,  for  each  S  G  Ca,  there  is  a  sentence  G 
L*(3)  such  that  ^4  |=  y>B  and  B  (pB>  Let  6  =  I\b^Ca  verify 

that  Mod/(^)  =  Mod/(Th^(A)).  ■ 

Kolaitis  and  Vardi  [12]  obtained  a  normal  form  for  the  negation  free  fragment 
of  Z/^4^(3)  over  IF.  It  is  easy  to  extend  their  result  to  and  to  provide  a 

dual  normal  form  as  well.  We  codify  these  normal  forms  in  the  next  proposition. 

Proposition  16  (Kolaitis  and  Vardi  [12]).  For  each  (p  G  L^f^{3),  there  is 
a  6  £  V(A^*(3))  a  C  €  A(V-^*^(3))  such  that  Mod/(<^)  =  Mod/(^)  = 
Mod;(C). 

Proof.  Let  C  =  Mod/(y?).  By  Proposition  10,  for  each  A  G  CyB  G  F  —  C, 
there  is  a  sentence  dA,B  G  ^*^(3)  such  that  A  [=  9a,b  and  B  0a,b-  Let 
^  =  VA€c(ABec  and  let  C  =  Ab{?c(Va€C  ^a,b)-  It  is  easy  to  verify  that 
the  proposition  holds  for  this  choice  of  §  and  C  ■ 

Next  we  show  that  the  fragments  /\L*(3)  and  YL*^(3)  are  closed  under 
finite  conjunction,  finite  disjunction,  and  existential  quantification  over  F.  This 
means  that  if  an  (3)-definable  query  cannot  be  expressed  in  either  /\L^{3) 
or  V  ■L*'(3),  then  it  is  only  definable  using  both  an  infinitary  conjunction  and  an 
infinitary  disjunction. 

Proposition  17.  The  languages  /\L*'(3)  and  \/  (3)  are  both  closed  under  fi¬ 

nite  conjunction^  finite  disjunction,  and  existential  quantification  over  F. 

Proof.  Let  ^  y)  |  f  G  w}  be  a  set  of  formulas  of  L*(3).  We  show  that  if 

9{y)  =  3a?y\^,  then  0(y)  is  equivalent  over  F  to  some  formula  9^{y)  G  AL^(3). 
(The  other  closure  conditions  may  be  easily  verified.)  Let  '4>m  = 
and  let  9'(y)  =  Am€w  We  show  9^  is  equivalent  to  0.  It  is  obvious  that  9 

implies  9'.  Let  A  £  F  and  a  G  A  be  such  that  A  [=  9' [a].  Because  A  is  finite, 
there  is  some  a'  £  A  such  that  for  arbitrarily  large  m,  A  (=  Therefore 

^  N  Amew  and  9'  implies  9.  ■ 

Below  we  show  that  the  query  classes  /\  L^(3)  and  V  L*(3)  are  proper  subsets 
of  /\{\J  L*^(3))  and  that  neither  of  y\L^(3)  and  V  L*'(3)  is  a  subset  of  the  other. 
We  first  give  necessary  and  sufficient  conditions  for  classes  to  be  definable  in 

ALH3)and  VL*(3). 
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Proposition  18.  1.  A  class  C  is  definable  in  /\L^{3)  iff  for  all  B  ^  C,  there  is 
a  (pB  ^  such  that  B)f^pB  and  for  all  A 

2.  A  class  C  is  definable  in  V  L^{3)  iff  for  all  A  E  C,  there  is  a  (p a  ^  ^^(3) 
such  that  A}^  Pa  and  for  all  B  ^  CyB  Pa- 

Proof.  To  prove  1.,  suppose  that  C  is  defined  by  the  sentence  Anet*' 

B  ^  C.  Then  there  is  some  V’m  such  that  B  ^  ipm-  Let  pB  be  this  In  the 

other  direction,  observe  that  the  sentence  p  =  Ab^c  defines  C.  The  proof  of 
2.  is  similar.  ■ 

Proposition  19.  For  each  k>2,  there  is  a  polynomial  time  computable  boolean 
query  C  €  f\L^{3) -y  L^{3). 

Proof.  Let  Jb  >  2  be  given  and  let  the  graph  Ak  be  a  model  of  the  fc-Gaifman 
theory.  Let  T  be  the  L*^(3)-theory  of  Ak  and  let  6  —  f\T.  Clearly,  B  E  l\L^{^)- 
Let  C  =  Mod;(^).  It  is  easy  to  see  that  C  =  {B  E  T  \  Ak:<^B}.  It  then  follows 
immediately  from  the  fact  that  the  relation  is  polynomial  time  computable 
(see  Kolaitis  and  Vardi  [12]),  that  C  is  polynomial  time  computable.  In  the  proof 
of  Proposition  13,  we  showed  that  for  every  satisfiable  p  E  L*'(3),  Mod/(v?)  2 
It  follows  immediately  that  C  ^  Mod/(^)  for  every  sentence  tp  E\/  L^{3).  ■ 

Proposition 20.  There  is  a  polynomial  time  computable  boolean  query  C  E 
y  L'^(3)  such  that  for  all  k  E  Uy  C  ^  f\L^{3).  In  consequence,  for  each  k  >  2, 
there  is  a  class  C  E  V L*(3)  —  f^L^{3). 

Proof.  Over  the  signature  a  —  {EySyt}yletC  =  {A  \  there  is  a  path  from  s  tot}, 
the  class  of  (s,f)-connected  graphs.  This  class  is  clearly  in  VL^(3).  As  noted 
earlier,  it  is  in  Datalog,  and,  hence,  polynomial  time  computable.  From  Proposi¬ 
tion  18, to  show  that  C  ^  L*'(3),  it  suffices  to  show  that  there  is  a  B  such 

that  for  all  n  G  w,  there  is  an  G  C  such  that  This  latter  condition 

is  equivalent  to  D’s  possessing  a  winning  strategy  for  the  n-round  3*-game  on 
An  and  B.  We  construct  B  to  give  her  the  greatest  possible  freedom  in  choosing 
her  moves.  Let  M  be  any  graph  such  that  M  (=  A+i,  and  let  Ms  (resp.  Mf)  be 
obtained  from  M  by  requiring  that  s  (resp.  t)  denote  a  loop-free  element.  We 
define  B  to  be  the  disjoint  union  of  M,  and  Mt,  thus  insuring  that  B  ^C. 

For  each  n,  let  An  be  the  simple  chain  from  s  to  f  of  length  2"'*'^.  The  basic 
idea  is  that  by  choosing  the  chain  to  be  long  enough,  S  will  not  be  able  to  witness 
the  existence  of  a  path  from  s  to  f  in  only  n  moves.  Let  d{xy  y)  be  the  natural 
distance  function  on  An- 

We  now  describe  D’s  strategy.  In  each  round  m,  D  chooses  to  play  on  an 
element  of  M,  iff  S  just  played  a  pebble  on  a  G  such  that  either  (i)  d(s,  a)  < 
2(n+2)-m.  Qj.  there  is  a  j  such  that  f3j  is  on  an  element  of  M,  and  d(aj,  a)  < 
2(n+2)-m  She  then  plays  her  pebble  on  an  element  of  the  appropriate  component 
of  B  so  that  she  maintains  a  partial  isomorphism  among  the  pebbles  on  that 
component.  It  is  easy  to  see  that  this  is  possible  because  M,  and  are  models 
ofA+i. 

In  order  to  establish  that  this  is  a  winning  strategy,  it  suffices  to  verify  the 
following  two  clairyis. 
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1.  In  each  round  /  <  n,  if  D  plays  a  pebble  /?,*  on  M,,  then  a,-  is  not  adjacent 

to  t  on  An.  Similarly  for  Mt  and  s. 

2.  After  each  round  I,  for  all  pairs  of  pebbles  {a,-,  ay},  if  An  |=  E{ai,  ay),  then 

ft  and  ft  are  on  the  same  component  of  B. 

We  argue,  by  induction,  that  if  D  plays  ft*  on  in  round  m,  then  d(s,  a,*)  < 
(2(n+2)-i  2(n+2)-2  ^  ^  2("+2)-'")  <  2"+*  -  1.  Since  d{s,t)  =  2"+\  this 

establishes  that  ^  E{ai,t),  In  round  1,  D  plays  ft-  on  M,  iff  d(s,  a,-)  < 
2(n+2)-i  Suppose  that  in  round  m  + 1  D  plays  ft-  on  Ms.  Then  either  d(s,  a,-)  < 
2(n4-2)-m  there  is  an  ay  such  that  ft-  is  on  M,,  d(a,-,  ay)  <  2(”+2)-(m+i)^ 
by  induction  hypothesis,  d{s,  ay)  <  +  . . .  +  2(”+2)“”^).  In 

both  cases,  the  induction  condition  is  maintained.  The  second  part  of  Claim  1 
follows  from  the  fact  that  in  round  m,  if  D  plays  ft-  on  Mt ,  then  S  must  have 
played  a^  such  that  d(s,a,-)  >  >  1.  To  prove  Claim  2,  observe  that  at 

each  round  m,  if  ft-  6  M,,  and  ft-  G  Mt,  then  d(aj,ay)  >  2^”+^)“"*  >  1.  The 
details  are  similar  to  the  previous  argument.  ■ 

The  next  result  shows  that  the  normal  form  for  L^^(3)  over  given  in 
Proposition  16  is  optimal. 

Proposition  21.  For  all  k>2,  there  is  a  class  C  C  suck  that  C  G 

(Ai*(3)uVi"(3)). 

Proof.  The  proof  of  this  proposition  is  a  synthesis  of  the  proofs  of  the  pre¬ 
ceding  two  results.  We  define  a  set  of  models  {Ai,  A2, . . .}  which  are  pairwise 
L*(3)-incompatible  such  that  for  each  i,  the  L*(3)-theory  of  A,-  is  not  finitely 
axiomatizable  in  ^**(3).  We  then  let  C  =  {B  \  3i{Ai:<^ B)}.  The  arguments  to 
show  that  this  class  is  neither  in  \J  (3)  nor  in  /\L^(3)  are  minor  variants  of 
the  proofs  of  Propositions  19  and  20. 

We  define  each  model  A,*  as  an  expansion  of  a  homeomorphic  image  of  a 
graph  which  is  a  model  of  the  {k  1)-Gaifman  theory.  Let  be  a  finite  graph 
that  satisfies  Taj+i;  observe  that  R  also  verifies  A.  Each  A,-  is  obtained  from  R 
by  replacing  all  edges  which  are  not  loops  by  pairwise  disjoint  paths  of  length  z. 
Where  there  is  a  two-way,  undirected  edge,  a  single  undirected  path  is  inserted, 
rather  than  two  directed  paths.  To  clarify  the  exposition,  we  also  add  a  unary 
predicate  V  to  the  signature  to  label  the  original  ‘vertices’  of  R. 

To  verify  that  C  is  not  in  \/  (3)  ^  it  suffices  to  show  that  there  is  a  model 
A  G  C  and  a  sequence  B^,B^,.,.,  disjoint  from  C,  such  that  for  each  n,  . 

Let  A  be  Ai ,  and  let  each  B^  be  obtained  from  the  model  B^  from  the  proof  of 
Proposition  13  by  putting  every  element  into  the  extension  of  the  predicate  V. 
From  that  proof  it  is  immediate  that,  for  all  n,  but  Aij^^B^.  For  all 

2  <  i.  A,-  \=  3x^Vx  and,  consequently,  Ai-^^B^.  This  establishes  that  each  5” 
is  not  in  C. 

In  order  to  show  that  C  0  /\L*'(3),  we  now  define  a  single  B'  ^  C  such 
that  for  all  n,  there  is  an  Ay(„)  such  that  Ay(„ 5'.  By  Proposition  18, 
this  will  establish  that  C  ^  /\L^{3).  Let  R"^  be  an  expansion  of  R  obtained 
by  labeling  exactly  one  looped  element  with  the  predicate  V;  and  let  R~  be 
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obtained  similarly  by  labeling  a  loop-free  element.  Here  the  predicate  V  plays 
the  same  role  as  the  constants  s  and  t  in  the  proof  of  Proposition  20.  We  define 
to  be  the  disjoint  union  of  k  copies  of  both  and  R~,  and  let  f(x)  =  2®+^. 
It  is  easy  to  see  that  B'  0  C.  As  in  the  proof  of  Proposition  20,  the  Duplicator 
wins  the  n-move  3*-game  on  A2ft+2  and  because  the  labeled  vertices  of  yl2n+2 
are  too  far  apart  for  S  to  distinguish  the  models  by  witnessing  that  they  are 
actually  connected.  ■ 

Finally,  we  prove  the  following  separation. 

Proposition  22.  Over  T,  for  k>S,  L^{3)  C  (A  H  (V 

Proof  Let  Path(a:,  y)  express  the  binary  query  ‘there  is  an  F^-path  from  x  to  y.' 
For  signature  cr  =  {E,s],  we  define  C  -  {A  \  3x{  Path(5,x)  and  Path(a;,  a?))}. 
Let  0n(x,y)  be  an  L^(3)  formula  that  defines  the  binary  query  ‘there  is  a  path 
of  length  n  from  x  to  y.'  It  is  easy  to  see  that  C  is  in  V  L*(3).  Also  observe  that 
P  —  Anew  3a;3t/(s  =  x  A  ^n(aJ,2/))  defines  C.  Finally,  there  are  arbitrarily  large 
minimal  models  in  C,  that  is,  models  A  £  C  such  that  for  all  proper  submodels 
B  C  A,B  ^  C.  This  immediately  implies  that  C  ^  FO(3)  and,  a  fortiori,  not  in 
1^(3).  . 

4  The  Failure  of  Existential  Preservation  for  L'^ 

OOU> 

In  this  section  we  prove  that  n  EXT  g  L^^(3).  Indeed,  we  establish  that 
there  is  a  sentence  0  G  such  that  Mod(0)  is  closed  under  extensions,  but 
there  is  no  ^  G  such  that  Mod/(^)  =  Modf(i/’):  Thus,  0  witnesses  the 

failure  of  existential  preservation  for  simultaneously  over  the  class  of  finite 
structures  and  over  the  class  of  all  structures.  The  central  lemma  on  which  this 
result  relies  is  of  interest  in  itself.  It  says  that  for  all  it  >  3,  the  finitary  language 
fails  in  a  strong  way  to  satisfy  an  existential  preservation  property.  Andreka, 
van  Benthem,  and  Nemeti  [3]  showed  that  for  every  k  ^  3,  there  is  a  sentence 
<Pk  E  which  is  preserved  under  extensions,  but  which  is  not  equivalent  to 
any  sentence  of  L*(3).  For  k  >3,  the  sentence  ipk  they  construct  uses  a  relation 
symbol  of  arity  k-l  and  has  the  property  that  it  is  equivalent  to  a  sentence  of 
L*+^(3).  They  state  the  following  open  problems. 

-  For  any  ^  >  3  and  n  G  w,  find  sentences  (pn  E  which  are  preserved  under 
extensions,  but  which  are  not  equivalent  to  any  sentence  of  ^^+*^(3). 

-  For  Ar  >  3,  is  there  a  formula  of  containing  only  (one)  binary  relation 
symbols  which  is  preserved  under  extensions,  but  is  not  equivalent  to  any 
sentence  of  L*(3)? 

The  next  proposition  settles  both  these  open  problems.  The  main  result  of  the 
section  follows  easily  from  the  proof  of  this  proposition. 

Proposition  23.  For  each  k  <  u,  there  is  a  sentence  9k  E  L^,  containing  a 
single  binary  relation,  such  that 


r 
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1.  Mod(^jfc)  is  dosed  under  extensions^  but 

2.  Mod/(^jb)  /  Mod/(^)  for  all  <p  £  (3). 

Proof.  Before  presenting  the  full  proof,  we  sketch  the  basic  outline.  Let  the 
k-pyramid  of  B,  V^{B),  be  the  smallest  class  of  (finite  and  infinite)  models 
containing  B  that  is  closed  under  substructures  and  L^-equivalence.  For  each 
^  >  3,  we  define  finite  structures  Ak  and  Bk  with  the  following  properties: 

1. 

2.  V^(Bk)  is  L^-definable; 

3.  Ak^V^Bk), 

Let  (pk  6  be  such  that  Mod(v?jb)  =  'P^{Bk)i  and  let  6k  =  -'(pk-  It  is  obvious 
that  Mod(^fc)  is  closed  under  extensions,  that  Ak  ^jb,  and  that  Bk  ^  6k. 
Suppose  (p  E  L*(3)  is  such  that  Ak  (=  <p.  Since  Ak:<^u^kj  this  implies  that 
Bk  1=  (p,  and  therefore  that  (p  is  not  equivalent  to  6k . 

We  define  structures  Ak  and  Bk  in  terms  of  simpler  submodels.  For  f  <ty  let 
the  [t,  fyflag^  F[<,  /],  be  the  directed  chain  of  length  i  with  one  additional  vertex 
attached  to  the  link.  That  is,  the  vertex  set  of  ,  /]  is  {0, 1,  1},  and 

the  edge  relation  is  {(i,  2  + 1)  |  f  <  <}  U  {(/,  <  +  !)}.  Ak  is  the  disjoint  union  of  the 
flags — ^[2)^+2,  A;-f  1],  F[2/;H-2,  fc+2], . . .,  F[2^+2,  2^+1].  Let  the  [A:,  j]-<ree, 
T[A:,  j],  be  the  tree  obtained  from  Ak  by  fusing  the  2*^  nodes  of  each  flag,  for  all 
2  <  j.  This  tree  has  height  2A;  +  2  and  the  node  at  height  j  has  outdegree  A:  +  1. 
Then  Bk  is  the  disjoint  union  of  the  k  trees —  T[A:,  0],  T[A;,  1], . . . ,  T[At,  At  —  1]. 

First  we  show  that  Ak-<^^Bk  by  describing  a  winning  strategy  for  D  in 
the  eternal  3^-game  on  Ak  and  Bk.  A  component  of  a  model  is  a  maximal 
connected  submodel.  Observe  that  every  component  of  Ak  is  embeddable  in 
every  component  of  Bk.  Call  a  component  of  either  Ak  or  Bk  vacant  at  round 
n  if  there  is  no  pebble  located  on  any  element  of  that  component  before  the 
players  make  their  moves.  We  consider  two  cases  of  moves  for  S.  First, 
suppose  that  in  some  round  n,  S  plays  pebble  a,*  on  a  vacant  component  A” 
of  Ak.  Since  there  are  only  k  pairs  of  pebbles,  and  since  pebble  /Si  is  not  on 
the  board,  there  is  a  vacant  component  B”  of  Bjt,  and  an  isomorphic  injection 
/in  :  A”  !->■  B” .  D  will  play  pebble  /?,*  on  hn{cxi).  In  the  other  case,  S  plays  on  a 
non- vacant  component  A” .  There  is  some  m  <  n  such  that  A”  has  been  occupied 
continuously  since  round  m  and  either  m  =  1  or  A”  was  vacant  at  round  m  —  1. 
Thus  A”  =  A”*,  and  there  are  previously  defined  B"*  and  hm.  D  now  plays  /?j  on 
hmioii).  By  this  condition,  every  pair  of  pebbles  (a?,/?/)  on  components  A”^  and 
B^  satisfies  the  condition  that  =  A  -  In  both  cases,  it  is  clear  that  D  has 

maintained  a  partial  isomorphism.  By  Proposition  8,  it  now  follows  immediately 
that  Ak:<l^^Bk. 

Next,  we  show  that  V^iBk)  is  definable  in  L^.  Consider  the  following  prop¬ 
erties: 

1.  A  contains  no  chains  of  length  >  2Ar  -f  2. 

2.  A  contains  no  .cycles  of  length  <  2A:  -|-  2. 
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3.  No  element  a  €  A  has  indegree  >  2,  that  is,  A  |=  -^3x3p3z(x  ^  y  A  Exz  A 
Eyz). 

It  is  easy  to  show  that  each  property  is  expressible  in  is  closed  under  substruc¬ 
tures,  and  holds  of  B*..  Erom  this  it  follows  immediately  that  each  B'  G 
possesses  all  three  properties.  Consequently,  every  member  oiV^(Bk)  is  a  forest 
consisting  of  directed  trees  of  height  <  2k  2. 

Next  we  note  the  following  facts: 

Lemma 24.  Let  A  and  B  be  the  disjoint  unions  of  components 
and  {Bi, . . . ,  Bn},  respectively.  For  k>Z,  A=^^^B  if  and  only  if  for  each  com¬ 
ponent  Ai  [Bf],  either  the  number  of  components  of  A  that  are  -equivalent  to 
it  is  equal  to  the  number  of  components  of  B  that  are  -equivalent  to  it  or  both 
numbers  are  >  k. 

This  result  can  be  proved  by  a  simple  pebble  game  argument. 

Lemma  25.  For  each  h,  and  each  k>Z,  up  to  equivalence  in  there  are  only 
finitely  many  trees  of  height  <  h. 

The  proof  proceeds  by  induction  on  h.  The  case  where  h  =  1  is  obvious.  Given 
a  tree  T,  call  a  proper  subtree  that  contains  a  node  i  of  height  1  and  all  of  its 
descendents  a  l-tree  of  T.  For  h>  1,  we  claim  that  two  trees  Ti  and  T2  of  height 
at  most  h  are  L*-equivalent  if  and  only  if  for  each  1-tree  V  CTi^  the  number  of 
1-trees  of  Ti  that  are  -equivalent  to  T'  equals  the  number  of  l-trees  of  T2  that 
are  B^'-equivalent  to  T\  or  both  numbers  are  >  k.  The  argument  is  just  like  the 
proof  of  the  preceding  lemma.  From  the  claim,  the  lemma  follows  immediately. 

Corollary  26.  For  each  h,  and  each  k  >Z,  up  to  equivalence  in  L*  there  are 
only  finitely  many  forests  of  height  <  h. 

This  is  an  immediate  consequence  of  the  preceding  lemmas. 

These  observations  establish  that  there  are  only  finitely  many  complete  L*- 
theories  that  are  satisfiable  in  V^(Bk).  Moreover,  each  such  theory  has  a  finite 
model.  By  [5],  every  such  theory  is  axiomatized  by  a  single  sentence.  Hence, 
if  we  let  (pk  be  the  disjunction  of  these  sentences,  we  have  Mod(y?fc)  =  V^{Bk) 
as  desired. 

Finally,  we  argue  that  Ak  ^  V^{Bk). 'By  the  definition  of  V^{Bk)i  for  every 

B'  G  'P^(Bfc),  there  is  an  m  G  w  and  a  sequence  (Bq,  Bi,  Bi, . . . ,  of 

structures,  with  Bjb  =  Eq  and  B'  =  Bm,  such  that: 

1.  For  all  1  <  2  <  m,  B<  C  B,>.i. 

2.  For  all  1  <  i  <  m,  Di  Ei. 

It  suffices  to  show  that  for  any  such  sequence,  Ak  cannot  be  embedded  in  any 
Ei.  Let  g  :  P^(Bjk)  {0, 1, . . .,  +  1}  be  the  function  such  that  g{D)  is  the 
meiximum  number  of  components  of  Ak  that  can  be  embedded  in  B  pairwise 
disjointly.  We  show  that  for  each  i  <  m,g(Ei)  <  + 1.  In  fact,  we  show  that  g  is 
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monotonically  decreasing  on  the  aforementioned  sequence.  Because  each  Di  is  a 
submodel  of  it  is  clear  that  g{Di)  <  g{Ei-i).  It  remains  to  establish  that 
g{Bk)  <k-\-l  and  that  g{Ei)  <  g{Di). 

Observe  that  any  embedding  of  a  flag  F[2k  +  2^  /]  into  a  component  C  of  any 
B'  G  V^{Bk)  must  map  the  root  of  the  flag  to  the  root  of  C.  This  implies  that 
no  two  flags  of  Ak  can  be  disjointly  embedded  into  any  such  component  and, 
since  Bk  has  only  k  components,  that  g{Bk)  <  ^  +  1. 

From  Lemma  24,  it  follows  that  every  Ei  can  be  obtained  from  Di  by  repeated 
application  of  the  following  three  operations.  First,  replace  some  component  with 
a  component  that  is  L^-equivalent  to  it.  Second,  add  a  disjoint  copy  of  a  tree 
that  is  L^-equivalent  to  at  least  3  components.  Third,  remove  a  component  that 
is  I/^-equivalent  to  at  leeist  3  other  components.  Thus,  it  suffices  to  argue  that 
no  such  operation  performed  on  some  B'  E  V^{Bk)  can  yield  a  B”  such  that 
g{B”)  >  g{B*).  It  is  obvious  that  removing  a  component  cannot  increase  the 
value  of  g. 

We  claim  that  it  suffices  to  consider  the  effect  of  the  other  two  operations  on 
components  of  height  =  2k-\-2.  If  trees  T  and  T‘  are  L^-equi valent,  then  they 
have  the  same  height.  Also,  no  component  F[2k  +  2,  /]  of  Ak  can  be  embedded 
in  any  tree  of  height  <  2k  +  2.  This  establishes  that  the  presence  of  shorter 
components  in  a  model  B  does  not  affect  the  value  of  g{B). 

Observe  that  for  all  trees  T  and  T'  such  that  T  T',  F[i,  /]  can  be  embed¬ 
ded  in  T  iff  it  can  be  embedded  in  T'.  This  is  because  the  following  property 
can  be  expressed  in  L^:  there  is  an  element  x  such  that  (i)  there  is  a  y  such  that 
there  is  a  path  of  length  /  from  y  to  x;  (ii)  x  has  outdegree  2;  (iii)  there  is  a  y 
such  that  there  is  a  path  of  length  t  —  f  from  x  to  y.  Over  trees,  this  property 
says  that  the  model  embeds  F[i,f].  Consequently  the  operation  of  replacement 
cannot  increase  the  value  of  g. 

It  remains  to  establish  that  adding  an  additional  component  to  a  model 
B'  E  V^(Bk)  does  not  change  the  value  of  g.  We  observe  that  Bk  has  the 
following  properties: 

1.  For  each  (2^ -|- 2)- chain  contained  in  Bk  there  is  at  most  one  j,0  <  j  <  k  —  1, 

such  that  the  link  of  the  chain  has  outdegree  >  1. 

2.  For  each  {2k  +  2)-chain  contained  in  Bk  there  is  at  most  one  j,k  -\- 1  <  j  < 

2k  +  such  that  the  link  of  the  chain  has  outdegree  >  1. 

These  properties  are  closed  under  substructures  and  L^-equivalence;  consequently, 
they  hold  of  every  model  B'  E  'P^(Bfc).  Let  Ci,  ^2,  and  C3  be  B^-equivalent  com¬ 
ponents  of  B'  of  height  2k  -j-  2.  The  above  argument  establishes  that  each  C,*  is 
either  some  F[2^  +  2,  /],  or  the  simple  {2k  +  2)-chain.  Let  B"  be  the  extension 
of  B'  obtained  by  adding  a  component  C4.  Observe  that,  in  fact,  all  four  com¬ 
ponents  must  be  isomorphic,  and  embed  at  most  one  isomorphism  type  of  flag. 
Therefore,  the  image  of  any  embedding  h  :  Ak  B^'  can  contain  vertices  from 
at  most  one  of  these  four  components.  This  demonstrates  that  fif(B')  =  g{B”), 
and  completes  the  proof.  ■ 

The  following  result  establishes  the  failure  of  existential  preservation  for  . 
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Theorem  27.  There  is  a  sentence  0  £  such  that  both 

1.  Mod(^)  25  closed  under  extensions. 

2.  For  all  <f>  e  Mod/(^)  /  Mod;(yi). 

Proof.  We  claim  that  it  suffices  to  show  that  for  each  k  €(j0  there  is  a  sentence 
9k  G  and  a  pair  of  finite  models  Ak  and  Bk  such  that 

1.  Mod(^A:)  is  closed  under  extensions. 

2.  Ak  1=  Ok  and  Bk  9k. 

3.  Ak'^^i^Bk. 

4.  For  all  j,Aj  \=  9k. 

Let  0  .  It  is  clear  that  9  is  closed  under  extensions  and  that  it  has  finite 

models,  since  it  is  true  in  each  Ak.  Suppose  that  ^  is  a  sentence  in  i^„(3)  such 
that  9  implies  (p.  Then  Ak  |=  V’,  and  therefore  Bk  \=  (p.  But  for  all  /,  Bj  9. 
Therefore,  Mod/(0)  ^  Mod/(v?). 

The  sentences  9k  and  the  models  Ak  and  Bk  from  the  proof  of  Proposition 
23  fail  to  meet  condition  4  because  for  j  <  k,  Aj  Y^  9k.  To  see  this,  observe  that 
Aj  will  always  be  a  submodel  of  Bk.  To  fix  this  defect,  it  suffices  to  construct 
A'kfBl.,  and  as  in  the  proof  of  Proposition  23  that  also  satisfy  the  additional 
condition  that,  for  all  j  and  k^  Aj  ^  In  order  to  accomplish  this,  we  add 

simple  ‘gadgets’  to  the  models.  Let  the  k-cycle,  Cjb,  be  the  graph  on  k  vertices 
whose  edge  relation  forms  a  simple,  directed  cycle  of  length  k.  Then  let  A'j^  and 
Bj^  be  obtained  from  Ak  and  Bk,  respectively,  by  adding  a  disjoint  copy  of  Ck- 
By  slightly  modifying  the  proof  of  Proposition  23,  we  can  show  that  Ak':^^^^Bk, 
and  that  there  is  a  6  satisfied  by  exactly  the  models  in  the  complement 
of  V^{Bf^)  such  that  A'j^  h  Finally,  it  is  easy  to  verify  that  for  j  k,  the 
j-cycle  cannot  be  embedded  in  any  B  £  V^iB^.)  and,  therefore,  Aj  ^  O^..  ■ 

5  Generalized  Preservation  Theorems  in  the  Finite  Case 

In  this  section,  we  prove  some  generalized  preservation  theorems  for  fragments 
of  FO.  Our  results  are  of  the  form 

L  n  EXT  C  L' 

for  certain  quantifier  prefix  classes  L  C  FO  and  L'  =  or  Datalog(^, -i). 

Recall  that  Tait  [18]  showed 


FOnEXTgFO(3), 

and  that  Gurevich  and  Shelah  [9,  10]  gave  examples  showing  that 

FO[V*3’^]nEXT  g  FO(3). 

Compton  observed  that 


FO[3*V*]nEXTCFO(3), 


r 
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which  shows  that  these  examples  are  best  possible  in  terms  of  quantifier  alter¬ 
nation  prefix  (see  [9]).  Kolaitis  and  Vardi  (see  [2])  observed  that  the  example 
of  Gurevich  and  Shelah  [9]  can  be  defined  in  Datalog(/,  -i).  Theorem  29  below 
establishes  that 

FO[3"V3]  n  EXT  C  Datalog(9^,  -i). 

It  follows  that  all  the  examples  in  the  literature  witnessing  the  failure  of  the  Los- 
Tarski  Theorem  in  the  finite  czise  are  definable  in  Datalog(^,  -i),  since  all  these 
examples  are  in  the  prefix  class  FO[3*V3]  (a  sequence  of  existential  quantifiers 
followed  by  one  universal  quantifier  followed  by  one  existential  qunatifier).  The 
next  theorem  establishes  a  slightly  more  general  result  with  L^^(3)  in  place  of 
Datalog(^,  -i). 

Theorem 28.  FO[3'^V3*]  n  EXT  C  L^^(3). 

Proof.  Let  (p  €  FO[3^V3*]nEXT.  That  is,  p  e  FO[3"V3"]  and  Mod/(^)  G  EXT. 
Let  C  =  Mod/(^).  We  proceed  to  show  that  C  G  L^j^(3).  By  Proposition  10,  it 
suffices  to  show  that  there  is  a  ^  such  that,  for  each  A  and  B  there  is 
a  9a,b  G  L^^(3)  such  that  A  |=  9a,b  and  B  ^  9a,b^ 

Let  <p  =  3x1  •  •  .Xiiy3z\  . .  where  ‘ip  is  quantifier  free,  and  let 

k  =  (we  suppose,  without  loss  of  generality,  that  i  >  0).  We  now  describe 

a  winning  strategy  for  S  in  the  eternal  3*^ -game  on  A  and  B  ^  for  A  €C  and  B  ^C, 
which  establishes,  by  Proposition  8,  the  existence  of  9a,b  G  LJo^(3)  with  the 
desired  properties.  There  are  two  stages.  Let  a  =  (ai, . . . ,  Uj)  be  a  sequence  of 
elements  of  A  such  that  A  ^  ^y3zxp{a,  y^J).  If  D  has  not  lost  after  h  rounds,  for 
h  <  i,S  plays  pebble  o/j+i  on  element  o/i+i .  If  S  has  not  won  after  i  moves,  and 
D  has  played  her  pebbles  on  6  =  (^>i, . . . ,  ^*)>  B  |=  3yVz-tip(b^y,'z)  (since 
B  <p). 

The  goal  of  the  second  part  of  S’s  strategy  is  to  force  D  to  play  a  peb¬ 
ble  on  some  element  6'  such  that  B  ^  Vz-i'0(^)  without  removing  any  of 
the  pebbles  ai, . . . ,  a,*  which  *fix  the  interpretation’  of  the  variables  xi, . , . ,  Xj 
on  both  A  and  B.  Regardless  of  the  element  a'  on  which  S  will  have  played 
his  corresponding  pebble,  A  (=  3'z'tp{a,a\J),  so  that  he  can  then  win  eas¬ 
ily.  In  order  to  describe  S’s  strategy,  we  first  define  a  sequence  of  subsets  of 
the  universe  of  B.  Let  Fq  —  {b^  \  b'  £  B  and  B  |=  V2-i^(6, 6',  z)}.  Observe 
that  B  ^  3t/Vz-«V’(5, 2/, z),  and  therefore  Fq  is  non-empty.  Given  Fo^.^.^Fm, 
if  (UKm  ^/)  n  6  =  0,  then  let  jB^+i  be  the  submodel  of  B  whose  universe  is 

{B  -  [j/<rr»  ^0-  Let  Fm+I  =  {6'  |  F  G  J3m+1  and  Bm-\-i  \=  '^-'fp{b,b\y)}.  For 
each  Bmi  since  Bm  Q  B,  we  have  that  Bm  |=  Vx3?/Vz'-i^(x,  y,  z).  In  particular, 
Bm  1=  3yYz-i'ip{b y  y,  z)  and  thus,  as  above,  Fm+i  is  non-empty.  Since  B  is  finite, 
there  is  some  n  such  that  Fn  H  6  /  0,  and  some  element  bf  G  Fn  Hb  pebbled 
by  /?/.  Then  B  is  partitioned  into  the  sets  To, . . . ,  We  also  have  that 

A  3z^(a, a/,z),  and  Bn  [=  Vz-iV^(6, 6/, z). 

The  Spoiler  can  win  by  executing  a  substrategy  that  compels  D  to  play  in 
sets  Fm  of  successively  smaller  index.  Let  c  be  a  sequence  of  elements  of  length 
j  such  that,  A  ^(a,  a/,  c).  S  plays  his  next  j  moves  on  this  sequence,  until  D 
makes  a  losing  move  or  plays  a  pebble  Pg  onto  an  element  in  ,  for  m  <  n  —  1 . 
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We  claim  that  one  of  these  two  possibilities  must  occur.  For  suppose  that  D 
plays  on  a  sequence  d  C  Bn.  Then  Bn  ^  ->^(6,  6j,  rf),  and  witnesses 

that  the  function  that  takes  a  *  Oj  +  c  to  6  ♦  6^  *  d  and  preserves  the  denotations 
of  constants  is  not  a  partial  isomorphism. 

Suppose  that  D  has  played  some  pebble  jSg  into  some  set  Fm .  By  the  same 
argument  as  above,  reusing  pebbles  {a,+i, .  ..,«*}  -  {a^},  S  can  either  win  or 
force  D  to  play  into  some  Fm^,  for  some  m'  <  m.  Iterating  this  procedure,  S  can 
force  D  to  play  into  To,  and  then  win  by  using  the  same  procedure  one  more 
time.  ■ 

We  remark  the  following  two  refinements  of  the  foregoing  theorem. 

1.  For  each  B  ^  C,  there  is  a  number  ms  such  that  for  all  A  e  C,  S  wins  the 

ms-round  B^'-game  on  A  and  B.  (Here,  ms  is  determined  by  the  maximum 
number  of  sets  F  that  get  defined  on  B,  for  any  choice  of  D^s  first  i  moves.) 
It  follows  easily  from  Proposition  7  that  this  condition  is  equivalent  to  there 
being  a.  6b  €  T*(3),  with  quantifier  rank  <  ms,  such  that  for  all  A  6  C, 
A\=  Osi  and  B  9b  >  Then  6*  =  As^c  is  equivalent  to  (p  and  is  a  single 
infinite  conjunction  of  T*(3)  sentences.  We  know  by  Proposition  20  that  not 
all  sentences  of  can  be  expressed  in  this  form.  Indeed,  it  follows  from 

Theorem  29  below  that  if  6  FO[3*V3]  D  EXT,  then  p  is  equivalent  to  a 
formula  in  /\L^{3)  D  V  L^(3)  for  some  k. 

2.  Suppose  that  (p  is  an  sentence  with  quantifier  type  V3*  (this  notion  of 
quantifier  type  may  be  defined  straightforwardly,  and  is  distinct  from  the 
notion  of  prefix  class).  In  this  case,  we  can  show,  by  a  modification  of  the 
proof  of  Theorem  28,  that  p  is  equivalent  to  an  T^^(3)  sentence.  This 
contrasts  with  Proposition  23  above  which  established  that  for  all  k,  there 
is  a  sentence  pk  G  such  that  Mod/(^jb)  G  EXT,  but  pk  is  not  equivalent 
over  F  to  any  sentence  in  LJqj^(3). 

Theorem  29.  FO[3*V3]  n  EXT  C  Datalog(#,  --). 

Proof.  Let  p  —  Bxi . .  .Xj'iy3zj3{x,  y,  z),  with  /?(x,  y,  z)  quantifier  free.  Let  c  = 
(ci,...,Cp)  be  the  sequence  of  constants  in  the  signature  of  p  and  let  C  = 
Mod/(^).  For  a  ^  A,  we  say  that  a  closes  with  parameters  a  iff  there  is  a 
sequence  Oo(=  a),  oi, . . . ,  a„  such  that  for  all  /  <  n,  A  [=  ^(a,  a/,  oj+i)  and  there 
is  an  m  <  n  such  that  A  \=  /?(a,  a„,a^).  Note  that  this  is  equivalent  to  there 
being  an  a'  such  that  there  is  a  ^(a,  y,  z)-path  from  a  to  a',  and  a  /?(a,  y,  z)-cycle 
including  o'. 

We  claim  that  A[=  p  if[  there  is  a  j-tuple  a  such  that  every  element  of  a  U  c 
closes  with  parameters  a.  Suppose  that  A  does  not  satisfy  these  conditions.  We 
prove  that  A  |=  V®3j/Vz“»y0(jr,  y,  z))  where  the  latter  sentence  is  equivalent  to 
-tp.  Let  a  C  be  a  sequence  of  length  j.  By  hypothesis,  there  is  an  a'  G  a  U  c 
such  that  a'  does  not  close  with  parameters  a.  Since  A  is  finite,  this  implies 
that  there  is  an  m  >  0  and  a  sequence  a'  =  Oq,  . . . ,  such  that  for  alH  <  m, 
A  \=  /?(a,  aj,  0/^.1 )  and  A  (=  Vz-i/?(a,  aj„,.z),  as  desired. 
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In  the  other  direction,  let  a  be  such  that  every  member  of  a  U  c  closes  with 
parameters  a.  Let  =  (a;io(=  a/i),  •  • . ,  and  th  =  (e/io(=  c/»), . . . ,  be 

sequences  witnessing  that  each  element  of  aUc  closes  with  parameters  a.  Let  B 
be  the  submodel  of  A  with  universe  (J^  U  (Jj  •  Then  it  is  easy  to  verify  that 
B  (p  and,  since  Mod/(^)  6  EXT,  it  follows  that  A\=  (p. 

The  following  program,  with  x  =  (xi,. .  ,,Xj),  computes  p: 

P(x,  y,  z)  < —  /?(^  y,  z) 

P{x,  y,  z)  < —  P{x,  y,  w),  P{x,  w,  z) 

Q  < —  P{^,xi,yi),P(^,yi,yi),...,P{x^Xj,yj),P{x,yj,yj), 

P{x,  Cl,Wl),  P(x,  Wi,Wi),  P(x,  Cp,  Wp),  P(x,  Wp,Wp) 

This  can  be  easily  converted  into  a  Datalog(/, -i)  program.  Let  j3(x,y,z)  = 
Vj  Si,  where  each  Si  is  a  conjunction  of  literals.  Replace  the  clause  P(x,  y,  z)  < — 
/?(£,  y,  z)  with  the  clauses  P(x,  y,  z)  < —  6i,  for  all  i,  ■ 

6  Conclusion 

In  this  section  we  discuss  some  open  problems  that  are  naturally  suggested  by 
our  investigations  and  we  present  some  further  results  bearing  on  the  problem 
of  preservation  under  homomorphisms  in  the  finite  case. 


6.1  Open  Problems 

The  first  and  most  obvious  question  is  the  extent  to  which  our  results  can  be 
generalized  from  fragments  of  FO  to  the  entire  language.  In  this  connection,  we 
restate  two  of  the  problems  mentioned  earlier  which  remain  open  in  light  of  our 
study. 

Problem  1.  Is  FO  D  EXT  C  Datalog(/,  -i)? 

Problem  2.  Is  FO  H  EXT  C  L^^(3)? 

Obviously,  a  positive  answer  to  the  first  of  these  questions  implies  a  positive 
answer  to  the  second.  Should  the  answer  to  these  questions  be  negative,  it  would 
be  of  interest  to  characterize  the  classes  FOnDatalog(:^,  -»)  and  in 

some  informative  way.  An  example  of  a  characterization  of  this  kind  is  the  follow¬ 
ing  theorem  of  Ajtai  and  Gurevich  [2].  FO'^(3)  denotes  the  positive  existential 
fragment  of  FO. 

PropositionSO  (Ajtai  and  Gurevich  [2]).  FO  fl  Datalog  =  FO’^(3). 

As  remarked  above,  the  Gurevich-Shelah  counterexample  to  the  Los-Tarski  The¬ 
orem  in  the  finite  case  witnesses  that  FO  fl  Datalog(/, -i)  /  FO(3).  Might 
FO  n  Datalog(^,  -i)  be  contained  in  some  level  of  the  first-order  quantifier  al¬ 
ternation  hierarchy,  be  it  not  the  lowest  level?  Should,  on  the  other  hand,  the 
answer  to  Problem  1  be  positive,  we  might  try  to  establish  even  stronger  results 
such  as  a  positive  answer  to 

Problem  3.  Is  (FO+LFP)  Pi  EXT  C  Datalog(7^,  -t)? 


501 


6.2  Preservation  under  Homomorphisms 

In  this  subsection  we  briefly  turn  our  attention  to  a  different  preservation  prop¬ 
erty.  A  homomorphism  from  A  to  B  is  a  map  h  :  A  B  such  that  for  all 
n-ary  relation  symbols  and  for  all  n-tuples  a  C  A,  if  A  |=  R{a),  then 

B  |=  R(h(a)).  A  class  of  models  C  is  closed  under  homomorphisms  iff  for  all  A 
and  B  such  that  there  is  a  homomorphism  from  A  to  5,  if  A  €  C,  then  B  E  C. 
Let  HOM  denote  the  set  of  classes  in  JF  that  are  closed  under  homomorphisms. 
A  sentence  <p  in  FO,  etc.  is  positive^  if  and  only  if,  it  does  not  contain  any 
negations.  The  following  well-known  classical  result  is  a  direct  consequence  of  the 
Los- Tarski  Theorem:  for  all  ip  G  FO,  Mod(^)  is  closed  under  homomorphisms, 
if  and  only  if,  (p  is  equivalent  to  a  positive  existential  sentence.  This  theorem 
is  one  of  a  few  classical  results  whose  validity  over  T  remains  unknown.  In  our 
current  notation,  we  can  formulate  the  question  as  the  following  open  problem, 
the  interest  of  which  has  been  emphasized  by  Gurevich  [10]  and  Kolaitis  (see 
[8]). 

Problem  4.  Is  FO  H  HOM  C  FO+(3)? 

(To  avoid  confusion,  it  should  be  remarked  that  although  [10]  announces  a  so¬ 
lution  to  Problem  4,  this  claim  has  been  withdrawn.) 

The  following  proposition  yields  some  information  about  the  homomorphism 
preservation  question.  We  direct  the  reader  to  [17]  for  its  proof. 

Propositional.  Datalog(/,  -.)  H  HOM  C  Datalog. 

Propositions  29,  30,  and  31  yield  as  an  immediate  corollary  the  following  special 
case  of  the  homomorphism  preservation  theorem. 

Corollary  32.  FO[3*V3]  D  HOM  =  FO+(3). 
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Abstract.  We  show  that  a  form  of  divide  and  conquer  recursion  on  sets 
together  with  the  relational  algebra  expresses  exactly  the  queries  over 
ordered  relational  databases  which  are  iVC-computable.  At  a  finer  level, 
we  relate  k  nested  uses  of  recursion  exactly  to  AC^,  k  >  1.  We  also  give 
corresponding  results  for  complex  objects.^ 


1  Introduction 

NC  is  the  complexity  class  of  functions  that  are  computable  in  polylogarithmic 
time  with  polynomially  many  processors  on  a  parallel  random  access  machine 
(PRAM).  The  query  language  for  NC  discussed  here  is  centered  around  a  divide 
and  conquer  recursion  {dcr)  on  sets  which  has  obvious  potential  for  parallel 
evaluation  and  can  easily  express,  for  example,  transitive  closure  and  parity,  dcr 
with  parameters  e,  /,  w  defines  the  unique  function  (p  =  dcr{e,  /,  u)  such  that: 

^(0)  e 

y’({j'})  '=  f{y) 

p{si  U  S2)  ^(52))  when  si  n  S2  =  0 

For  parity,  we  take  e  false ^  f{y)  true  and  u{vijV2)  vi  xor  V2.  To 
compute  the  transitive  closure  of  some  binary  relation  r,  take  e  0,  f{y)  r 

and  u(ri,  r2)  ri  U  r2  U  ri  o  r2.  Then,  the  transitive  closure  of  r  is  obtained  by 
applying  (p  to  the  set  of  nodes  of  the  relation  r,  namely  i^c(r)  =  y?(77i(r)Ui72(r’)), 
where  77i,  772  are  the  relational  projections.  In  general,  dcr(e,  /,  u)  is  well-defined 
when  there  is  some  set  containing  e  and  the  range  of  /,  on  which  u  is  associative, 
commutative  and  has  the  identity  e.  For  parity,  this  is  the  set  B  of  booleans, 
while  for  transitive  closure,  it  is  the  set  {r  U  U  . . .  U  I  ^  ^  0}- 

*  The  authors  were  partially  supported  by  NSF  Grant  CCR-90-57570  and  ONR  Con¬ 
tract  N00014-93-11284 

^  These  results  were  first  announced  at  PODS’94  [30]  whose  proceedings  contain  a 
somewhat  more  detailed  exposition  than  the  one  below.  A  journal  version  is  in  prepa¬ 
ration.  Here  we  are  giving  just  enough  definitions  to  clarify  the  statement  of  the  main 
results.  See  also  http :  //www .  cis . upeim .  edu/ an/dbl . html 
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We  show  that  dcr  together  with  the  relational  algebra  expresses  exactly  the 
queries  over  ordered  databases  of  flat  relations  that  are  iVC-computable.  We  also 
show  that  a  bounded  version  of  dcr  together  with  the  nested  relational  algebra 
expresses  exactly  the  queries  over  ordered  databases  of  complex  objects  that  are 
TV'C'-computable.  In  fact,  we  prove  the  more  refined  versions  that  relate  k  nested 
uses  of  (bounded)  dcr  exactly  to  the  subclass  AC^  of  NC  where  ib  >  1.  Some 
explanations  are  in  order: 

-  Computable  queries  are  in  the  sense  of  Chandra  and  Harel  [10],  naturally 
extended  to  complex  objects. 

-  Any  language  that  can  express  the  class  of  queries  expressed  by  first-order 
logic  would  do  just  as  well  as  the  relational  algebra.  Similarly  for  complex  ob¬ 
jects,  where  a  corresponding  class  of  tractable  queries  has  emerged  from  sev¬ 
eral  equivalent  formalisms.  Some  of  these  formalisms  are  syntactically  restricted 
higher-order  logics,  others  are  algebraic  languages,  often  called  nested  relational 
algebras,  hence  our  phrasing  above. 

-  dcr  and  (nested)  relational  algebra  have  meaning  over  any  (nested)  rela¬ 
tional  databeise.  But,  as  with  all  known  characterizations  of  query  complexity 
classes  below  NP,  we  know  how  to  capture  the  entire  TVC  only  over  ordered 
databases.  Formally,  we  do  this  by  extending  the  language  with  an  order  predi¬ 
cate. 

-  A  bounded  version  of  dcr  is  necessary  over  complex  objects,  otherwise 
queries  of  high  complexity  such  as  powerset  will  be  expressible.  The  bounded 
version  is  obtained  by  intersecting  the  result  with  a  bounding  set  at  each  re¬ 
cursion  step  (section  2).  This  is  similar  to  the  bounded  fixpoints  studied  in  [29], 
and,  as  with  fixpoints,  over  flat  relations  dcr  can  always  be  expressed  through 
bounded  dcr  (section  2). 

We  believe  that  these  results  are  of  interest  from  two  angles. 

Z  Query  language  design,  dcr  is  a  well-known  construct.  It  appears  un¬ 
der  the  name  pump,  in  a  language  specifically  designed  for  a  parallel  database 
machine,  FAD  [4].  Following  FAD,  but  under  the  name  horn,  it  was  included 
in  Machiavelli  [24]  where  it  fit  nicely  into  the  language’s  type  system.  Called 
(a  form  of)  transducer,  it  is  part  of  SVP  [27],  precisely  in  order  to  support  di¬ 
vide  and  conquer  parallelism.  Some  limitations  of  its  theoretical  expressive  power 
were  examined  (under  the  name  horn)  hy  Immerman,  Patnaik,  and  Stemple  ([22] 
theorem  7.8).  They  also  note  that  dcr  is  in  NC. 

As  part  of  a  larger  group  of  researchers,  we  became  interested  in  dcr  because 
it  fits  into  a  natural  hierarchy  of  query  languages  that  share  a  common  seman¬ 
tic  basis,  built  around  forms  of  structural  recursion  on  collection  types  [7,  6,  8] 
(see  section  2).  Theoretical  studies  of  expressiveness,  such  as  [33,  6,  29]  and  the 
present  paper  help  us  with  the  choice  and  mix  of  primitives,  as  well  as  implemen¬ 
tation  strategies.  In  particular,  dcr  is  at  the  core  of  a  sublanguage  for  which  we 
are  currently  seeking  efficient  implementation  techniques  for  a  variety  of  parallel 
architectures. 

Z  Computational  complexity.  Following  Vardi  and  Immerman ’s  influen¬ 
tial  result  [32,  18]  that  first-order  logic  with  least  fixed  point  captures  exactly 
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the  PTIME-comp\iiah\e  queries  on  flat  relations  over  ordered  databases,  several 
characterizations  of  low  complexity  classes  in  terms  of  logics  or  algebras  used 
in  databases  have  been  discovered  with  the  hope  that  logical  methods  may  give 
insights  into  the  diificult  problem  of  complexity  class  separation.  We  mention 
first  a  few  of  these  characterizations  which  have  had  a  direct  influence  on  the 
work  here. 

For  parallel  complexity  classes,  Immerman  [21]  shows  that  the  class  of  finite 
and  ordered  relational  structures  recognizable  in  parallel  time  f(n)  (n  is  the  size 
of  the  structure)  on  a  certain  CRCW  (concurrent  read  -  concurrent  write)  PRAM 
coincides  with  the  class  of  structures  definable  by  a  first-order  induction  [23] 
of  depth  up  to  t{n).  For  complex  object  databases,  Grumbach  and  Vianu  [16, 
15]  give  a  syntactic  restriction  of  the  ramified  higher-order  logic  CALC  which, 
together  with  inflationary  fixpoints  and  in  the  presence  of  order,  captures  exactly 
the  PT/AfF^- computable  complex-object  queries.  Suciu  [29]  shows  that,  in  the 
presence  of  order,  the  same  class  of  queries  is  captured  by  the  nested  relational 
algebra  augmented  with  an  inflationary  bounded  fixpoint  operator. 

To  the  best  of  our  knowledge,  no  characterization  of  parallel  complexity 
classes  of  queries  over  complex  objects  has  been  given  before.  What  is  more 
likely  to  set  our  results  apart,  however,  is  the  intrinsic  nature  of  the  language 
we  are  proposing:  the  semantics  of  dcr  puts  it  naturally  in  NC\  there  is  no  need 
to  impose  logarithmic  bounds  on  the  number  of  iterations  or  recursion  depth. 
Moreover,  it  can  be  shown  that  a  different  kind  of  recursion  on  sets,  namely  struc¬ 
tural  recursion  on  the  insert  presentation  of  sets  ([7];  notation  sri;  definitions 
reviewed  in  section  2),  together  with  the  relational  algebra  expresses  exactly 
the  P  TIME -computable  queries  on  ordered  databases.  This  follows  from  results 
in  [22];  we  state  the  corresponding  result  for  complex  objects  in  proposition  8. 
Hence,  at  least  over  ordered  databases,  the  difference  between  NC  and  PTIME 
boils  down  to  two  different  ways  of  recurring  on  sets,  divide  and  conquer  vs. 
element  by  element. 

Gurevich  [17]  and  Compton  and  Laflamme  [12]  characterize  the  DLOGSPACE- 
and  respectively  the  AC  ^-computable  global  functions  on  finite  and  ordered  rela¬ 
tional  structures  as  algebras  with  certain  primitive  recursion  schemas.  Compton 
and  Laflamme  capture  NC^  also  with  first-order  logic  augmented  with  BIT.  and 
with  an  operator  for  defining  relations  by  primitive  recursion.  The  kinds  of  recur¬ 
sions  used  in  these  two  papers  are  very  different  from  dcr  because  they  depend 
on  some  linear  ordering  of  the  underlying  structures  for  their  actual  definition. 
While  dcr  is  a  form  of  recursion  on  finite  sets,  these  recursions  are  on  notations 
for  elements  of  (linearly  ordered)  finite  sets.  Of  course,  we  do  not  attempt  to 
characterize  DLOGSPACE  or  NC^  or,  for  that  matter,  any  class  below  AC^, 
but  see  Immerman ’s  characterizations  of  such  classes  in  terms  of  languages  more 
in  the  spirit  of  ours  than  of  those  of  Gurevich,  Compton,  and  Laflamme  [20,  19]. 

We  should  also  mention  here  the  work  of  Clote  [11]  for  related  characteriza¬ 
tions  of  most  parallel  complexity  classes  of  arithmetical  functions. 

We  should  point  out,  however,  one  sense  in  which  our  language  is  not  as 
neat  as,  say,  first-order  logic  with  least  fixpoint,  which  captures  PTIME  in  the 
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presence  of  order,  or  first-order  logic  with  transitive  closure,  which  captures 
NLOGSPACE  in  the  presence  of  order.  For  dcr  to  be  well-defined,  the  opera¬ 
tions  involved  in  it  must  satisfy  certain  algebraic  identities  (associativity,  com¬ 
mutativity,  identity)  and  this  turns  out  to  be  an  undecidable  condition  (in  fact 
Hi  complete;  see  section  2).  Of  course,  only  a  certain  family  of  instances  of  dcr 
is  actually  needed  in  the  simulations,  and  for  these,  the  algebraic  conditions 
always  hold.  Hence,  it  is  of  theoretical  interest  that  there  is  a  decidable  sublan¬ 
guage  of  dcr  plus  relational  algebra  which  captures  exactly  NC  in  the  presence 
of  order.  In  practice,  we  have  found  it  useful  to  provide  special  syntax  for  some 
instances  of  dcr  in  which  the  algebraic  conditions  are  automatically  satisfied, 
but  we  found  it  counterproductive  to  limit  dcr  to  these  instances,  as  other  uses 
kept  appearing. 

2  Recursion  on  Sets 

A  fruitful  approach  to  the  design  of  query  languages  for  complex  objects  is  to 
consider  tuples  and  sets  as  orthogonal  [6,  8].  Hence,  there  will  be  primitives  that 
work  on  tuples,  primitives  that  work  on  sets,  and  general  primitives  for  combining 
other  primitives.  In  this  section  we  discuss  several  choices  for  defining  recursions 
on  sets  and  the  relationship  between  them.  One  of  these  forms  of  yrecursion — 
dcr — will  be  at  the  core  of  our  “query  languages  for  AC” .  A  precise  definition 
of  the  languages  will  be  given  in  section  3. 

Complex  objects  are  built  essentially  from  tuples  and  finite  sets.  To  describe 
them,  we  define  the  complex  object  types  by  the  grammar: 

t  1=  D  I  B  I  unit  I  f  X  f  I  {f} 

where  D  is  some  base  type,  B  is  the  type  of  booleans  and  unit  is  the  type 
containing  only  the  empty  tuple  (unit  =  {()}).  The  values  of  type  sxt  are  pairs 
(x,  y)  with  X  E  s^y  and  the  values  of  type  {i}  are  finite  sets  of  elements  from 
t.  Products  of  types  of  the  form  {s},  with  s  a  product  of  base  types  (D,  B,  unit)^ 
are  called  flat  types.  We  will  also  need  to  consider  function  types  having  the 
form  s  where  s  and  t  are  complex  object  types. 

There  seem  to  be  two  basic  ways  of  describing  the  structure  of  finite  sets. 
In  one  way,  they  are  generated  by  finitely  many  (maybe  zero!)  binary  unions 
of  singleton  sets.  We  call  this  the  union  presentation.  In  another  way,  they  are 
generated  by  finitely  many  insertions  of  one  element,  starting  with  the  empty 
set.  We  call  this  the  insert  presentation.  Recognizing  the  relevant  algebraic  iden¬ 
tities  satisfied  by  union  (associativity,  commutativity,  idempotence,  has  0  as  an 
identity)  and  by  element  insertion  (i-commutativity  and  i-idempotence)  gives  us 
two  different  algebraic  structures  on  finite  sets.  Both  these  algebras  are  charac¬ 
terized  by  universality  properties,  which  amount  to  definitions  of  functions  by 
structural  recursion  [7,  6].  We  have  a  structural  recursion  construct  on  the  union 
presentation,  sru: 

e  :t  f  :  8  ^t  u  :t  xt  —^t 
sru{e,  fyu)  :  {s} -i- 1 
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sra(e,/,u)(0)  e 

sru{e,f,u){{y})  f{y) 

sru{e,  f,  u){si  U  *2)  ='  u(sni(e,  /,  «)(si), 

«ni(e,  /,  «)(S2)) 

sru(e,/,  If)  is  well-defined  when  there  is  some  subset  of  <  containing  e  and  the 
range  of  /,  on  which  u  is  associative,  commutative,  idempotent,  and  has  the  iden¬ 
tity  e.  We  also  have  a  structural  recursion  construct  on  the  insert  presentation, 
sri{x/'s)  is  the  element  insertion  operation,  {x}  U  s): 

e  :t  i :  s  xi 
sn(e,  i)  :  {s}  — >  t 

sn(e,f)(0)  1=  e 

sri{ej){y/'s)  i{y,  sri{e,i){s)) 

sri{e,  i)  is  well-defined  when  i  is  i- commutative,  and  i-idempotent  {i{x,  i{y,  s))  = 
i(y,  i[x,  s)),  i{x,  i{x,  s))  =  i(x,  «))  on  some  subset  of  t  containing  e. 

The  divide  and  conquer  recursion  dcr,  whose  definition  was  already  given  in 
section  1,  is  superficially  related  to  sru.  For  example  sru{e,f,u)  is  well-defined 
then  so  is  dcr{e,  f,  u)  and  they  are  equal.  But  dcr  is  potentially  more  expressive, 
since  u  need  not  be  idempotent.  In  fact,  we  do  not  know  if  sru  can  express  parity 
or  transitive  closure.  One  can  prove: 

Proposition!,  sri  can  express  dcr,  which  in  turn  can  express  sru  [7,  30].  All 
these  are  done  with  at  most  polynomial  overhead. 

Over  complex  objects  dcr  can  express  powerset  hence  we  need  some  restric¬ 
tion  if  we  are  to  stay  within  NC.  An  analog  to  Peter  Buneman’s  idea  of  bounded 
fixpoints  [29]  does  the  job.  We  define  a  PS-type  (product  of  sets)  to  be  either 
a  set  type,  or  a  product  of  PS-types.  The  bounded  version  of  dcr  is  defined  by: 

e:t  f:s—^t  u:txt—^t  b:t 
bdcr(e,  /,  if,  b)  :  {s}  — »•  t 

where  t  is  a  PS-type,  with  the  semantics: 

bdcr{e,  f,u,b)  1=  dcr{enb,  f  nb,unb) 

(here  {u  fl  b){y,  i/)  if(y,  ?/)  fl  6,  etc).  As  for  dcr,  we  define  the  bounded  sri, 
bsri{e,i,  b),  for  some  PS-type  t,  to  be  sri{enb,  in6).  Proposition  1  easily  extends 
to  the  bounded  versions  of  recursion.  Over  flat  relations  the  explicit  bounding 
is  unnecessary: 

Proposition 2.  bdcr  together  with  the  relational  algebra  can  express  dcr  when 
its  arguments  are  flat  relations  and  its  values  are  of  flat  PS-type.  Similarly  for 
bsri  and  sri. 
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Unfortunately,  for  a  language  at  least  as  expressive  as  first-order  logic,  ver¬ 
ifying  most  of  the  identities  required  by  the  sn,  dcr,  and  srw  constructs  is  as 
hard  as  testing  the  validity  of  a  first-order  formula  in  all  finite  models,  hence  it 
is  a  ijy-complete  question. 

3  Query  Languages 

In  this  section  we  define  the  languages  that  offer  characterizations  of  NC  over 
complex  objects  and  over  flat  relations.  For  complex  objects,  the  language  is 
obtained  by  adding  hdcr  and  an  order  predicate  to  the  nested  relational  algebra 
This  name  can  be  used  to  describe  a  language  that  has  the  same  expres¬ 
sive  power  as  Schek  and  Scholl’s  NF^  relational  algebra  [28],  or  Thomas  and 
Fischer’s  algebra  [31],  or  Paredaens  and  Van  Gucht’s  nested  algebra  [25,  26],  or 
Abiteboul  and  Beeri’s  algebra  without  powerset  [2].  The  particular  formalization 
we  use  here  was  essentially  introduced  in  [8]  and  is  based  on  what  is  called  there 
the  monad  calculus. 

For  each  type  s  we  assume  an  infinite  set  of  variables  to  be  given,  and  write 
ar*  for  a  variable  of  type  s.  As  usual,  we  distinguish  between  free  and  bound 
variables,  and  we  identify  those  expressions  that  differ  only  in  the  name  of  some 
bound  variables.  M'RA  is  defined  by  the  rules  in  figure  1. 


— e2  :  t2 
:  t  (ei,e2)  :  (^i,<2) 
e  :ti  xi2  e  :ti  xt2 

7ri(e)  :  ii 
e  :  t 


0:0} 


{e}  :  {*} 
ei  :  D  62  :  D 
=  62  :  B 
{t] 


7r2(e)  :  t2 
ei  :  {t]  ei  :  {t} 
ei  U  e2  :  {t} 


B 


0  :  unit 
ei  :i  t2  't 


empty{e)  :  B 
e  :  / 


if  e  then  ej  else  62  :  t 
f  :  3  —i^t  e  :  3 


Xx\t 


f:3 


/(e) :  t 

0} 


ext{f)  :  {*}  {«) 


Fig.  1.  The  Nested  Relational  Algebra 


We  briefly  describe  the  semantics  of  the  expressions:  (61,62)  constructs  a 
tuple,  7ri,7r2  are  the  projections,  {e}  is  the  singleton  set,  empty{e)  returns  true 


509 


iff  e  =  0,  z/  e  then  ei  else  e^  equals  ei  iff  e  =  true  and  62  otherwise,  Xx^  .e 
denotes  a  function  whose  input  is  the  variable  ar*,  /(e)  is  function  application, 

and  ext{f){{xij . . . ,  ar„})  f{^i)  U  . . .  U  /(a:„).  A  possible  set  E  of  external 

functions  p  ;  dom{j>)  — >  codom{p)  could  be  added  to  the  language;  in  this 
case,  we  denote  the  language  by  M'JtA{E).  We  abbreviate  with  X{x^y).e  the 
expression  \z,e[T:\{z)lx^'K2(z)ly]. 

Nil  A  is  powerful  enough  to  express  the  following  functions:  set  difference, 
set  intersection,  cartesian  product,  database  projections,  equalities  at  all  types, 
selections  over  predicates  definable  in  the  language,  nest  and  unnest  [8]. 

We  will  make  two  extensions  to  AflZA  in  order  to  obtain  a  language  that 
characterizes  NC.  One  extension  is  the  addition  of  divide-and- conquer  recursion 
on  sets,  dcr  is  too  powerful  on  complex  objects  since  it  can  express  powerset 
(actually  one  can  show  that  Af7lA{dcr)  has  the  same  expressive  power  as  Abite- 
boul  and  Beeri’s  algebra  [2]).  So  instead,  we  will  add  the  bounded  version  of  this 
kind  of  recursion  and  consider  the  language  XfHAlbdcr).  We  shall  have  occa¬ 
sion  to  contrast  this  languages  with  Af'JlA{bsri),  Note  that  proposition  1  states 
that  XflZA{bdcr)  C  Xf'JZA{bsri)^  and  this  holds  even  in  the  presence  of  external 
functions. 

The  second  extension  concerns  ordering.  All  known  characterizations  of  com¬ 
plexity  classes  below  PTIME  are  about  ordered  databases,  and  ours  is  no  ex¬ 
ception.  We  isolate  the  use  of  ordering  into  the  addition  of  an  external  function 
<:  D  X  D  — v  B,  which  is  understood  always  to  denote  a  linear  order  on  D.  The 
order  relation  can  be  lifted  to  all  types. 

We  have  thus  reached  the  language  A/’7^^(6dcr,  <)  which  is  the  subject  of 
one  of  our  main  theorems,  characterizing  the  AC-computable  queries  on  ordered 
databases  of  complex  objects  (theorem  3). 

In  the  same  theorem,  we  obtain  a  finer  characterization,  involving  the  AC- 
hierarchy,  for  which  we  need  the  notion  of  the  depth  of  recursion  nest¬ 
ing  depf/i(e),  of  some  expression  e.  We  define  this  to  be  to  be  the  maximum 

depth  of  recursions  occurring  in  e.  For  example,  for  bdcr:  depih{bdcr{ej  /,  it))  1=^ 
Tnax(depih(e),  depik(f)A  depih{u))  (only  u  is  actually  iterated).  We  denote 
Xf7lA{bdcr^^^)  the  restrictions  of  the  language  Af'JlA{bdcr)  to  iteration  depth 
<  k.  We  have  thus  obtained  the  hierarchy  of  languages  MllA(bdcr^^\  <)  which 
is  the  subject  of  the  finer  characterization  given  in  theorem  3. 

To  characterize  the  AC- computable  queries  over  ordered  fiat  relations  (theo¬ 
rem  4),  we  denote  by  N'RA^  the  restriction  of  MHA  to  types  of  “set  height  <  1” , 
i.e.  the  only  types  allowed  in  MltA^  as  inputs,  outputs  and  intermediate  types 
are  products  of  base  and  fiat  types.  Its  expressive  power  is  essentially  the  same 
as  that  of  the  relational  algebra  [6],  hence  first-order  logic.  We  make  the  same 
two  extensions  as  for  M'RA,  except  that  here  it  is  safe  to  use  dcr.  For  good  mea¬ 
sure,  proposition  2  states  that  it  doesn’t  matter:  NTlA^{bdcr)  =  M'llA^{dcr). 
Note  however  that  this  fails  in  the  presence  of  certain  external  functions.  Again 
we  will  have  occasion  to  contrast  M%A^{dcr)  with  Af'flA^{sri)  so  note  that 
proposition  1  states  that  Af'ltA^{dcr)  C  N'JlA^{sri),  and  this  holds  even  in  the 
presence  of  external  functions.  Hence,  the  languages  of  interest  for  flat  relations 
are  AfllA}{dcr^  <)  and  the  hierarchy  MltA^ {dcr^^\  <). 
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4  Main  Results 

Assuming  some  enumeration  of  the  base  type  D  to  be  given,  we  encode  com¬ 
plex  objects  as  strings  over  the  alphabet  A  =  {0, 1,  {,},(,),  comma,  6/anfe}. 
The  order  of  elemenets  in  the  sets  is  irrelevant,  but  no  duplicates  are  allowed. 
E.g.  the  object  {(a,  (6,  a)),  (c,  (a,  a)),  (a,  (6,  c))}  could  be  encoded  as  the  string 
{1,  (10, 1),  (11,  (1, 1)),  (1,  (10, 11))},  if  the  encoding  of  D  assigns  1, 2, 3  to  a,  b,  c 
respectively.  Blanks  may  be  arbitrarily  scattered  in  an  encoding,  but  not  insided 
the  binary  numbers.  Furthermore,  we  make  a  second  encoding  of  complex  ob¬ 
jects  as  binary  strings  (i.e.  in  {0, 1}*)  by  replacing  each  of  the  8  symbols  in  A 
with  a  3-bits  binary  number. 

Recall  that  AC^  is  the  class  of  functions  {0, 1}*  — >•  {0, 1}*  computable  by 
a  “uniform”  family  of  circuits  of  polynomial  size  and  of  depth  0(log*'  n)  [13]. 
We  define  Q-NC  and  CQ-NC  to  be  the  class  of  queries  over  base  types  and 
flat  relations,  and  complex  objects  respectively,  which  are  in  NC .  Similarly,  we 
define  the  subclasses  Q-AC^  and  CQ^AC^ . 

Theorems.  A/"'7^>l(6dcr, <)  =  CQ-NC.  More  precisely,  for  every  k  >  1  we 
have  ^rnAibdcr^^\<)  =  CQ-ACK 

Theorem 4.  {dcr ,  <)  =  Q-NC.  More  precisely,  for  every  k  >  I  we  have 

J\fnA\dcr^^\<)=  Q-ACK 

These  languages  are  purely  for  complex  objects,  respectively  relations.  But 
many  external  functions  of  practical  interest  such  as  the  usual  arithmetical  op¬ 
erations  (-b,  etc),  and  the  usual  aggregate  functions  (cardinality,  sum, 

average,  etc.)  are  also  in  NC.  Can  they  be  added  in?  The  answer  is  given  by: 

Propositions.  Lei  E  be  an  extension  consisting  of  possible  additional  base 
types  and  a  set  of  functions  computable  in  NC.  Then  /f'R.A{Eybdcr)  C  NC. 
However,  A/’7^w4^(N, -f,  dcr)  can  express  exponential  space  queries. 

Immerman  in  [21]  and  Barrington,  Immerman  and  Straubing  in  [5]  prove  that 
FO  is  included  in  FO-DCL-umfoim  AC^,  and  that  FO  together  with  order  and 
BIT  relation  has  the  same  expressive  power  as  Here,  we  prove  that  M1ZA 
is  included  in  A (7°,  thus  extending  half  of  their  results  to  complex  objects. 

Proposition 6.  All  queries  inMTLA{^,  are  in  FO-DCL-uniform  AC^  (see  [5]). 

We  state  two  more  results  which  help  us  put  the  main  theorems  in  perspec¬ 
tive.  Their  proofs  are  omitted  from  this  extended  abstract. 

Conservative  extension.  One  may  wonder  in  what  sense  theorem  4  is  a 
“particular  case”  of  theorem  3.  Actually,  even  though  the  proof  of  theorem  4 
is  quite  similar  to  that  of  theorem  3,  theorem  4  in  fact  follows  from  theorem  3, 
proposition  2  and  the  conservative  extension  result  presented  below. 
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Paredaens  and  Van  Gucht  in  [26],  and  Wong  in  [33]  prove  that  Nil  A  is  a 
conservative  extension  of  NUA^ .  Suciu  in  [29]  proves  that  NllA{hfix)  is  a  con¬ 
servative  extension  oi  NUA^ {fix),  where  fix  is  the  usual  inflationary  fixpoint, 
and  bfix  is  a  bounded  version  of  fix.  Using  the  techniques  in  [29],  we  can  prove 
the  following: 

Proposition  7.  Lei  L  be  a  set  of  external  functions  which  have  set  heights  <  1. 
Then,  NllA{TI ,  bdcr ,  <)  is  a  conservative  extension  of  NUA^  {S ^  bdcr ,  <) . 

Note  that  for  the  case  when  =  0,  we  can  turn  the  tables  and  proposition  7 
follows  directly  from  the  main  theorems.  For  the  case  when  /  0,  this  propo¬ 
sition  requires  a  separate  proof,  and  we  are  only  able  to  do  it  in  the  presence  of 
order.  However,  we  conjecture  that  NllA{bdcr)  is  a  conservative  extension  of 
NllA\dcr). 

PTIME  vs.  NC.  Immerman,  Patnaik  and  Stemple  [22]  show  that  PTIME 
is  captured  by  a  language  built  around  set-reduce  (see  section  2).  Extending  their 
result  also  to  complex  objects  we  have: 

Propositions.  NUA^{sri,<)  =  NllA^{sri^^^ ,<)  =  PTIME  [22]  and,  for 
complex  objects,  NllA{bsri,  <)  =  N'}lA{bsri^^\  <)  =  PTIME. 

Thus,  by  the  main  theorems  and  this  proposition,  the  difference  between 
PTIME  and  NC  computable  queries  over  ordered  databases  can  be  characterized 
by  the  difference  between  two  kinds  of  recursion  on  sets.  It  is  interesting  to  note 
that  only  one  level  of  recursion  nesting  suffices  for  sri  and  PTIME,  as  opposed 
to  dcr  and  NC. 


5  Conclusions 

Ordering  seems  to  play  a  crucial  role  in  capturing  complexity  classes  below 
i\rp  and  our  characterization  is  no  exception.  Indeed,  it  follows  from  theorem 
7.8  in  [22]  that  in  the  absence  of  ordering  FO  +  dcr  cannot  express  the  lower 
bound  in  [9]  which  is  in  AC^  plus  parity  gates  ([9]  remark  7.2).  As  with  PTIME, 
DLOGSPACE ,  etc.,  it  remains  an  important  open  question  whether  there  exists 
an  r.e.  set  of  “programs”  that  express  exactly  the  AC-computable  queries  over 
arbitrary  relational  databases. 

On  the  other  hand,  studying  the  expressiveness  of  the  various  forms  of  re¬ 
cursion  on  sets  in  the  absence  of  ordering  is  quite  relevant  to  query  language 
design.  It  may  also  be  relevant  to  complexity  theory,  if  an  analog  to  the  surpris¬ 
ing  result  of  Abiteboul  and  Vianu  [3]  holds.  They  have  shown  that  PTIME^ 
PSPACE  iff  first-order  least  fixpoint  queries  first-order  while  queries.  (Vardi 
had  shown  that  in  the  presence  of  order  the  FO-\- while  captures  PSPACE  [32].) 
Dawar,  Lindell,  and  Weinstein  [14]  give  a  machine-independent  proof  of  the 
Abiteboul  and  Vianu  result  making  use  of  properties  of  bounded  variable  log¬ 
ics.  Abiteboul,  Vardi  and  Vianu  [1]  give  evidence  for  the  robustness  of  the  idea 
with  several  such  results  for  other  pairs  of  complexity  classes.  In  our  case,  the 
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analog  would  be:  NC  /  PTIME  iff  FO  +  dcr  ^  FO  +  sri  (in  our  formalism, 
MTZA  (dcr)  ^  ^TZA  (sn)).  By  setting  aside  the  ordering,  with  its  potential 
for  tricky  encodings,  this  would  strengthen  the  observation  (section  4)  that  the 
difference  between  tractable  sequential  and  tractable  parallel  computation  can 
be  characterized  as  the  difference  between  two  ways  of  recurring  on  sets. 
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